Click Here to view the repository
Repository

Web Scraping Weather and News Data Analysis

  • Objective

    Showcase proficiency in web scraping techniques using Beautiful Soup and Pandas, focusing on extracting and analyzing data from a website about Mars weather patterns and news articles.

    Methods

     Part 1: Scrape Titles and Preview Text from Mars News  Automated browsing to visit the Mars news site.  Utilized Beautiful Soup to create a structured object for parsing HTML.  Extracted titles and preview text of news articles, storing data in Python dictionaries and lists.  Part 2: Scrape and Analyze Mars Weather Data  Automated browsing to visit the Mars Temperature Data Site.  Employed Beautiful Soup to scrape data from an HTML table.  Transformed scraped data into a Pandas DataFrame with appropriate headers and data types.  Conducted data analysis using Pandas functions to address specific queries about Martian weather

    Findings

      Extracted and analyzed data from Mars news articles and weather patterns.   Uncovered trends in Martian weather conditions, such as temperature and atmospheric pressure variations across different months.   Estimated the duration of a Martian year in terrestrial days.

    Impact

     Provided valuable insights into Martian weather patterns and trends, beneficial for scientific research and space mission planning.  Demonstrated effective utilization of web scraping techniques for real-time data extraction and analysis.

    Challenges

     Addressed challenges in parsing HTML elements and managing missing data during web scraping.  Ensured data integrity and accurate conversion of data types for precise analysis.

    Tools Used

     Terminal  Jupyter lab  Anaconda  HTML  css  Python  Splinter  Matplotlib  Pandas