Proxies for AI training data
Building a dataset from the public web means crawls that run for weeks across several languages. StarProxy unlimited residential lets you fix that cost on day one and send requests from 60 countries.
Per GB billing is hard to budget here
Across thousands of domains, page size is out of your hands. A handful of sites with heavy images and scripts can push GB use several times past the estimate.
Unlimited is priced only by period and connection tier. You can write the cost of a 90 day crawl as a single number before the first request.
Keeping languages apart
Many sites pick a language edition from the visitor's IP country. Add a country code such as -cc-fr (France) or -cc-br (Brazil) to the username and requests exit there.
Give each language its own job queue and a different username. The crawler code stays the same, and results land in a separate folder per language.
Example: cost of a 90 day crawl
The 200 connection, 100 Mbps tier on Season (90 days) is $3.46 a day, $311.40 in total. Buying Day 90 times would be $409.50, so Season saves $98.10.
For a larger crawl, the 1,000 connection, 200 Mbps tier on Season is $7.52 a day, $676.80 in total. The dashboard also takes any period from 1~365 days, so you pay for your crawl schedule and nothing more.
Ground rules for collection
Collect public content only and follow robots.txt and site terms. Check the copyright and privacy rules for training use, and strip personal data you do not need before storing anything.
Set a per domain request cap even when the proxy has spare capacity. You will want to revisit the same sites for the next dataset version.
Setting up StarProxy for AI training data
1
Top up in crypto to cover the full crawl period. One top-up can be up to $5,000.
2
Buy unlimited residential for the number of days your crawl runs. Season (90 days) has the lowest daily rate.
3
Create one username per language, such as spexample1-zone-unl-cc-jp, and attach each to a job queue.
4
Connect the crawler to gate.starproxy.shop:6100 and keep concurrent connections within the tier you bought.
Common questions
Can one plan use several countries?
Yes. One unlimited plan covers all 60 countries. You set the country per request in the username.
Does a crawl need sticky sessions?
Mostly not. A new IP per request spreads the load evenly. Use -sid only on sites where cookies have to carry across several pages.
Which coins can I top up with?
The accepted coins and networks are shown in the payment window. The amount goes into your USD balance.
What if balance is left after the crawl?
Balance has no expiry. Use it on the next crawl or on any other product.
Top up $25, get 69 GB of residential traffic
Pick a username and a password and you are in. Your balance never expires.