Parameters

apiKey

  • Header or Body
  • String
  • Required

An API key is a required parameter that may be passed in the header or body of your request. You can read more about this parameter in the Authentication section of the documentation.

pageUrl

  • Body
  • String
  • Required

The pageUrl should be a valid URL and is a required field. It should be passed through the body of your request as a string. Be sure you're able to load the URL in a browser to confirm it's valid and loads correctly.

Status Code

The URL passed must return a HTTP Status 200 header when loaded by our service.

Content Type

The content-type loaded from the URL you pass to the API must be HTML, an image, or a PDF file. If you are loading an image or PDF file, you should also pass the prop unprocessed with a value of true.

unprocessed

  • Body
  • Boolean
  • Optional

Use the unprocessed parameter to receive the raw response from a URL. The parameter is useful when loading images or PDF files. You can also receive unprocessed HTML from a URL to handle data extraction on your own. In some cases, you may also need the full HTML response for use cases, such as change monitoring.

mapping

  • Body
  • Object
  • Required | Optional if unprocessed true

The mapping parameter specifies what elements to grab values from after fetching the HTML and also controls the formatting of the JSON response.

The keys of the request object are used as the keys in the response JSON. Response data is returned in the data key of the response. This gives you great flexability to specify the format of your JSON response.

There are two ways to grab data from a page.

Single Element

A single element on a page can easily be grabbed from a page and return by using a "keyName": ".selectorFormat" format. The title key below is a good example of this. The title value of the page is grabbed and returned by passing "title": "head>title" in the mapping object.

Multiple Repeating Elements

There will be cases when you want to grab data that repeats. A simple example of this would be weather data. A 10-day weather forecast will contain 10 sets of high and low tempetures.

To grab this repeating data we will specify a selector key to grab the parent container. We will then use the foreach key to specify specific elements that repeat on the page.

Take a close look at the complete example below. You can see how the title of the page is grabbed by using the single selector format. You can also see how the forcast key returns multiple values from the page by using selector to specify the parent container along with foreach to loop over repeating values found within the parent container such as the high and low tempetures of a 10-day forcast.

Example mapping Request Object
Example Response JSON

We recommend creating a basic mapping by first specifiying your structure and key names. You'll need to identify what data will have a single value and what data repeats. Based on this you'll use one of the two selector formats found above. Keep in mind selector and foreach keys are reserved within the mapping parameter. This means you should never try to name a key in your response using either of those proteced keywords.

Once you've created your basic object, you'll want to work on your selectors.

Anything API uses a powerful selector system that closely parallels jQuery selectors. This allows you to grab data from a page in many different ways. You may be surprised by how creative you can get with selectors to grab the data you need.

ssr

  • Body
  • Boolean
  • Optional

Pass ssr as a parameter in the body if you need server-side rendering. Only use server-side rendering if you need it for a JavaScript-based website. Using ssr slows down the response due to the additional overhead to first load the page and then render it on the server.

countryCode

  • Body
  • Boolean
  • Optional

countryCode can load localized data from one of twelve countries. This can be helpful when trying to pull localized shipping data, prices, regional media, translations, etc. Keep in mind a request to a server in another country will add network latency due to the extra travel time between servers. This should not be substantial, but responses will take longer to return.

The default country code is us if the parameter is not provided. To pull data from another country, use one of these twelve country code values.

  • United Statesus
  • Canadaca
  • United Kingdomuk
  • Germanyde
  • Francefr
  • Spaines
  • Brazilbr
  • Mexicomx
  • Indiain
  • Japanjp
  • Chinacn
  • Australiaau

sessionId

  • Body
  • Integer
  • Optional

Easily reuse the same session for multiple requests with the sessionID param. The value can be any integer between 1 and 9,999. Sending the same integer within 60 seconds will recycle your session. Sending a new number will create a new session.