Oxylabs

Ai Crawler

Tool useful for crawling a website from starting url and returning data in a specified format. Schema is required only if output_format is json, csv or toon. 'render_javascript' is used to render javascript heavy websites. 'return_sources_limit' is used to limit the number of sources to return, for example if you expect results from single source, you can set it to 1.

Ai Scraper

Scrape the contents of the web page and return the data in the specified format. Schema is required only if output_format is json or csv. 'render_javascript' is used to render javascript heavy websites.

Ai Browser Agent

Run the browser agent and return the data in the specified format. This tool is useful if you need navigate around the website and do some actions. It allows navigating to any url, clicking on links, filling forms, scrolling, etc. Finally it returns the data in the specified format. Schema is required only if output_format is json, csv or toon. 'task_prompt' describes what browser agent should achieve

Ai Search

Search the web based on a provided query. 'return_content' is used to return markdown content for each search result. If 'return_content' is set to True, you don't need to use ai_scraper to get the content of the search results urls, because it is already included in the search results. if 'return_content' is set to True, prefer lower 'limit' to reduce payload size.

Generate Schema

Generate a json schema in openapi format.

Ai Map

Tool useful for mapping website's URLs.

Universal Scraper

Get a content of any webpage. Supports browser rendering, parsing of certain webpages and different output formats.

Google Search Scraper

Scrape Google Search results. Supports content parsing, different user agent types, pagination, domain, geolocation, locale parameters and different output formats.

Amazon Search Scraper

Scrape Amazon search results. Supports content parsing, different user agent types, pagination, domain, geolocation, locale parameters and different output formats. Supports Amazon specific parameters such as category id, merchant id, currency.

Amazon Product Scraper

Scrape Amazon products. Supports content parsing, different user agent types, domain, geolocation, locale parameters and different output formats. Supports Amazon specific parameters such as currency and getting more accurate pricing data with auto select variant.