Grab Proxy list Using Unix Command

use CURL cmd to copy web page, then GREP, CUT, and SORT to scrape ip proxies list
Repeat for each page of proxies

curl http://www.proxy4free.com/page1.html > proxy1.html

grep whois\.cgi\?domain\=proxy1.html | cut -d \= -f 3 | cut -d \" -f 1 | sort | uniq> proxy.txt

Alter accordingly for different sites when site alters page formatting

Using a few Linux/Unix commands (ports also available for Windows) the proxy lists can be downloaded, parsed, and formatted into something that you should be able to integrate into your Internet filtering solution. Remember that sites can change their formatting and proxy list sites can come and go. The key point here is not that you can use the list from proxy4free.com in your blocking (although you certainly can) but that you can make use of these resources with a minimal amount of effort to provide much better site blocking.

Suka artikel ini? Bagikan :

FacebookGoogle+