3+1 Chrome Tools you can use to scrape websites faster

Photo by Jens Kreuter on Unsplash

1. The copy functionality

Copy XPath directly from the Chrome Developer Tools
// So by copying the XPath for the element we get a query that will work for the specific element
// But our aim is to get all cite_note-* elements so we'll need to experiment with that.
//*[@id="cite_note-4"]

2. The $x command

// In our case it helps us find a way to match all cite_notes and make sure that our XPath syntax is correct.
$x('//li[contains(@id,"cite_note-")]')
This way we’re getting all 75 matching elements.

3. Copying requests (Reproducing complex requests in your code)

Copy the request as a cURL command

4. Curl to code service (Github)

Founder & CTO @ GuestFlip Connect with me: https://arispattakos.carrd.co/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store