Effortless Proxy Management With Scrapoxy
Seamlessly integrate Scrapoxy for easy proxy management and IP rotation, ensuring private browsing and secure, scalable web scraping.
55M+ IP pool
Advanced targeting
195+ global locations
24/7 tech support
Easy to integrate
99.99% uptime
What is Scrapoxy?
Scrapoxy is a powerful proxy aggregator that enables centralized management of multiple proxies within a single interface. It generates a pool of private proxies from multiple proxy services, intelligently routes traffic, and handles proxy rotation for you.
User-friendly interface
Simplify the management of proxy instances with an intuitive user interface, making it easy to set up, configure, and monitor proxies efficiently.
Ban management
Overcome bans by injecting proxy names into HTTP responses, allowing scrapers to notify Scrapoxy to remove problematic proxies from the pool when bans are detected.
Traffic monitoring
Monitor traffic, measure incoming and outgoing data, and track key metrics like request counts, active proxy numbers, and requests per proxy.
Why residential proxies?
A residential proxy serves as a mediator, allowing users to get an IP address from an authentic desktop or mobile device connected to a local network. Due to its origin, residential proxies are a perfect match for overcoming geo-restrictions, bypassing CAPTCHAs, managing multiple accounts, and conducting web testing with the CapSolver platform.
Smartproxy offers top-notch residential proxies with an extensive IP pool of over 55M IPs across 195+ locations. With an unparalleled responsive rate, clocking in at under 0.5 seconds, a success rate of 99.68%, and an affordable entry point with Pay As You Go, Smartproxy is an excellent deal for hustlers and fast-growing companies.
Set up Smartproxy proxies with Scrapoxy
To get started, you'll need 3 things: Docker, Scrapoxy, and proxies. Follow each step below to set everything up.
Getting residential proxies
Log in to your Smartproxy dashboard, find residential proxies by choosing Residential under the Residential column on the left panel, and select a plan that suits your needs. Then, open the Proxy setup tab and copy the username and password. Save it, as you'll need to use it later.
Installation
To use Scrapoxy, you'll need to install Docker first. Then, follow these steps:
- Launch the Docker Desktop application.
- Open the Settings menu.
- Go to the General tab.
- Enable Docker terminal.
- Click Apply & restart.
- Launch the Terminal from the bottom-right corner and enter the following command (replace admin, password, secret1, and secret2 with your custom values):
docker run -d -p 8888:8888 -p 8890:8890 -v ./scrapoxy:/cfg -e AUTH_LOCAL_USERNAME=admin -e AUTH_LOCAL_PASSWORD=password -e BACKEND_JWT_SECRET=secret1 -e FRONTEND_JWT_SECRET=secret2 -e STORAGE_FILE_FILENAME=/cfg/scrapoxy.json fabienvauchelles/scrapoxy
7. Navigate to http://localhost:8890 and log in with your username and password you provided in the command just earlier (not the proxy authentication information).
You can also use NPM. Check out the official documentation for instructions on how to set it up.
Configuration
After logging in, you'll be prompted to create a new project. Enter the following information and select options:
- Name. Create a custom name for your project.
- Minimum proxies. Set the minimum amount of proxies that should be used in this project.
- Auto Rotate Proxies (ms). If enabled, proxies will randomly rotate within a provided interval.
- Auto Scale Up. If enabled, all proxies will be started upon sending a request.
- Auto Scale Down. If enabled, all proxies will be stopped when no requests are sent.
6. Intercept HTTPS requests with MITM. If enabled, Scrapoxy will intercept and modify HTTPS requests and responses.
7. Keep the same proxy with cookie injection. If enabled, Scrapoxy will inject a cookie to maintain the same proxy for a single browser session.
8. Override User-Agent. If enabled, the header will be overridden with the value assigned to a proxy instance. This ensures all requests within the instance have the same User-Agent header.
9. Shuffle TLS Ciphersuite. If enabled, Scrapoxy will assign a random TLS cipher suite to each proxy instance to avoid TLS fingerprinting.
10. Once you're done setting up, click Create.
The project is set up, and you'll need to choose a provider from the Marketplace. Find Smartproxy in the list and click Create.
In the next step, enter the credentials for Smartproxy:
- Name. Create a custom name for the credentials. This can help differentiate between them if you're using several products.
- Product. Select the product you're using from the dropdown. In our example case, select Residential or Mobile.
- Username. Enter the username you previously saved from the Smartproxy dashboard.
- Password. Enter the password you previously saved from the Smartproxy dashboard.
- Click Create to save the credentials.
The final step is to create a Connector:
- Credentials. Select the credentials that you've just created previously from the dropdown list.
- Name. Enter a custom name for the Connector.
- # Proxies. The maximum number of proxies that the connector can provide and that you intend to use. You can adjust this later.
- Proxies Timeout. Set the time to attempt to connect to a proxy server before considering it offline.
- Proxies Kick. If enabled, set the maximum duration for a proxy to be offline before being removed from the pool.
- Country. Select a specific country to use proxies from or leave it at All.
- Session Duration (min). Set the duration of the session.
- Click Create.
- Once created, you'll see it in the list of Connectors. Enable it by toggling the Start/Stop this connector button.
Now that everything's ready, you can integrate Scrapoxy into your code and see how it works. Here's an example of how to use Python together with the requests library:
import requestsurl = 'http://ip.smartproxy.com/json'username = 'user'password = 'pass'ca = 'scrapoxy-ca.crt'proxy = f"http://{username}:{password}@localhost:8888"result = requests.get(url, proxies = {'http': proxy, 'https': proxy}, verify=ca)print(result.text)
Replace the username and password with the credentials from the project Settings in the Scrapoxy interface.
You must also download the CA certificate from Settings and place the file in the same directory as the script or write a full path to it in the ca variable. The endpoint should always be localhost:8888 unless configured otherwise.
To view and manage your proxies, you can use the Proxies, Coverage, and Metrics tabs in Scrapoxy. These provide information about active proxies, what countries they're located in, upload and download speeds, the number of requests, and their status.
For more information, check out the official documentation.
Configurations & Integrations
Easy Smartproxy proxy setup with popular applications and free third-party tools. Check out these guides to get started right away.
Buy residential proxies for any demand
What people are saying about us
We're thrilled to have the support of our 85K+ clients and the industry's best
Best online Customer support I've gotten
Customer Support is amazing, agent walked me through an issue I've been dealing since...
We almost forgot we're using proxy
Many regions/configurations available. Convenient API. Very reliable -- issues happen...
Perfect and reliable proxy service
I was in need of a proxy to pass some hard country IP location check and Smartproxy g...
Featured in:
Explore our other proxy line products
Residential Proxies
from $2.2/GB
Real, physical device IPs that provide a genuine online identity and enhance your anonymity online. Learn more
ISP Proxies
from $0.35/IP
IPs assigned by Internet Service Providers (ISPs), offering efficient and location-specific online access with minimal latency. Learn more
Mobile Proxies
from $4.5/GB
Mobile device based IPs offering anonymity and real user behavior for mobile-related activities on the internet. Learn more
Datacenter Proxies
from $0.09/IP
Remote computers with unique IPs for tasks requiring scalability, fast response times, and reliable connections. Learn more
Site Unblocker
from $1.6/1K req
A powerful application for all proxying activities offering dynamic rendering, browser fingerprinting, and much more. Learn more
Smartproxy Blog
Build knowledge on residential proxies, or pick up some dope ideas for your next project - our blog is just the perfect place.
Most recent
How to Scrape Amazon ASIN
Imagine you want to collect ASINs (Amazon Standard Identification Numbers) for all the products that appear on Amazon after searching for a specific item. This can be incredibly useful for tasks like market research, competitor analysis, or managing your own product listings. With our Amazon ASIN scraper, you can easily gather these ASINs directly from the search results, making the data collection process quick and efficient. In this guide, we’ll show you how to use our ready-made Amazon scraper to extract ASINs and explain how this information can benefit your business.
Dominykas Niaura
Dec 03, 2024
7 min read
Most popular
Video: Best Proxies for YouTube - No More Restrictions
With proxies, you can experience YouTube like never before. Unblocking geo-restricted Youtube content, scraping the platform, and using bots to automate your processes are just a few things proxies enable you to do.
Martin Ganchev
Feb 19, 2024
2 min read
Video: The Different Proxy Types
A simple explanation of different proxy types that will help you choose the best one for your use case.
Vilius Sakutis
Feb 19, 2024
2 min read
Video: Getting Data From Search Engines - Smartproxy SERP Scraping API
Looking for an all rounded search engine scraping tool? Well, have you tried Smartproxy’s SERP Scraping API? In this video you will find out what it is, what it can scrape and why it might be the only search engine scraping tool you’ll ever need.
Vilius Sakutis
Feb 19, 2024
2 min read
How to Bypass CAPTCHAs: The Ultimate Guide 2024
So, there you are, casually surfing the net, when… a CAPTCHA appears out of the blue, interrupting your flow. Yes, it’s that little test making sure you’re not a robot, and let’s face it – it can really slow down your processes. The great news? You don’t have to be stuck. It’s possible to bypass CAPTCHAs. So, buckle up, and let’s dive into the tricks that make these roadblocks the past.
Martin Ganchev
Dec 05, 2023
10 min read
How To Scrape Google Search Results, Or Rising To The Google Challenge [VIDEO]
Whenever you want to find an answer to a tricky question or dig out some advice, who (or what) do you approach first? Let’s be honest, it’s Google. Market research, competitor analysis, latest news, exclusive deals on designer clothing – whichever you’re after, 9 times out of 10, you’ll google it.
Being the richest encyclopedia in the world, Google is also the most protective of all search engines, so extracting data from it can be pretty hellish. On the bright side, there’s a way out. This tutorial will demonstrate how you can successfully scrape the world’s largest library by using Smartproxy’s SERP Scraping API.
Ella Moore
Dec 22, 2021
10 min read
Frequently asked questions
What is Scrapoxy?
Scrapoxy is a powerful proxy aggregator that manages and rotates proxies, helping developers scale web scraping operations while minimizing the risk of bans.
What is proxy scraping?
Proxy scraping involves using proxies to collect data from websites without revealing the scraper's real IP address, allowing for anonymous and distributed web scraping while avoiding blocks or rate limits.