Spade strives to be a simple and elegant service that takes away the cruft that you deal with when you want to scrape, or reuse, existing Web content.
Feel free to drop us a line at email@example.com and tell us about your experience with the docs.
Scraping: The Problem
We spend countless hours trying to grab content from the Web to build new products that aim to re-model information, extract insights and create better experience based on content that’s out there.
We’re used to view-source’ing Web pages and building custom scrapers, but then the worst happens: the layout changes.
Spade: The Solution
Spade will throw away useless bits such as ads, irrelevant images, irrelevant text and HTML pieces.
We aim to intelligently turn nonstructured mess into a semantic object structures that you can use with a simple API call.
Our stack is built on Scala, Clojure, and Node.js. Feel free to shoot us an email if you want to know more.
For the API transport protocol, we examined ZeroMQ, Thrift and plain HTTP. We chose HTTP for simplicity.