Resources#
EOT Dataset Access#
End of Term Datasets: Primary resource with EOT datasets and documentation.
EOT S3 Bucket Registry Entry: Main S3 bucket with associated open data registry entry.
Tutorials#
End of Term Parquet Workshop by Mark E. Phillips and Sawood Alam (2024): Step-by-step guide on analyzing the EOT WARC data in Parquet format.
Background Articles#
End of Term Web Archive Dataset: Longitudinal Web Archive of .GOV and .MIL Domains by Mark E. Phillips, Kristy Phillips, and Sawood Alam (2024): Overview of dataset characteristics, file formats, intended uses, and methodology.
Content-Based Characterization of the End of Term Web Archive by Mark E. Phillips, Kristy Phillips, and Sawood Alam (2023): Covers derivative creation and metadata management, including the technologies used and introduction of the WARC Metadata Sidecar for storing auxiliary metadata.