Web Scraping and Scripting with Python

Online Campus

Online
Anywhere
Online

Past Locations for this Workshop

Web Scraping and Scripting with Python | Online

Online Campus

Online
Anywhere
Online

Past Locations for this Workshop

About this workshop

Have you ever been on a web site and seen a listicle that has 50+ items in it, and in order to get through the list, you are forced to tediously click on the “next” button over and over? Have you ever wanted to know the average price of all AirBnB rentals in your neighborhood?

In this course, you’ll learn how to write Python scripts to programmatically retrieve and store (web scrape) data from websites like BuzzFeed and Esquire. In the process, you will also learn about basic data structuring and Chrome Developer Tools for investigating a website's HTML format.

Optionally, you'll also learn how to integrate info from an external API to enrich your scraped data.

Takeaways

  • Define web scraping and know why Python is an excellent tool to programmatically extract data from websites.
  • Scrape web pages with Python.
  • Format and store the scraped data as a CSV.

Prereqs & Preparation

Students should have completed the Intro to Python workshop, or have some beginner/intermediate experience in using Python. Completion of Learn Python the Hard Way before the workshop is recommended.

Preparation:

  • Please bring a laptop with your choice of text editor installed (recommend: Sublime).
  • Python2.7+ already installed along with the following Python modules: Beautiful Soup, requests, and ipython (all modules should be importable/functioning with your chosen version of Python).
  • The majority of the course will be taught via the ipython interpreter (not standard python interpreter), thus a working installation of ipython interpreter is required.
  • Google Chrome is advised for developer tools. If you are not familiar with Developer Tools, you should complete Chapters 1-2 of this tutorial (http://discover-devtools.codeschool.com/)

Coming up near you

Let’s Keep You Updated

Enter your email to start following

I have read and acknowledge General Assembly's Privacy Policy and Terms of Service. SMS message and data rates may apply.