Integrating Proxies with Python

Python is one of the most popular languages for web scraping and automation. Here's how to integrate HypeProxy.io proxies into your Python projects.

Using the requests Library

The requests library makes proxy configuration straightforward:

import requests

proxies = {
    'http': 'http://username:password@proxy-host:port',
    'https': 'http://username:password@proxy-host:port',
}

response = requests.get('https://api.ipify.org', proxies=proxies)
print(f'Your proxy IP: {response.text}')

With SOCKS5

Install the SOCKS dependency first:

pip install requests[socks]

Then use the SOCKS5 protocol:

proxies = {
    'http': 'socks5://username:password@proxy-host:port',
    'https': 'socks5://username:password@proxy-host:port',
}

response = requests.get('https://api.ipify.org', proxies=proxies)

Using aiohttp (Async)

For high-performance async scraping:

import aiohttp
import asyncio

async def fetch(url):
    proxy = 'http://username:password@proxy-host:port'
    async with aiohttp.ClientSession() as session:
        async with session.get(url, proxy=proxy) as response:
            return await response.text()

result = asyncio.run(fetch('https://api.ipify.org'))
print(f'Your proxy IP: {result}')

Using the HypeProxy.io API

Rotate your IP programmatically between scraping sessions:

import requests

API_TOKEN = 'YOUR_API_TOKEN'
PROXY_ID = 'YOUR_PROXY_ID'
HEADERS = {'Authorization': f'Bearer {API_TOKEN}'}

# Rotate IP
requests.get(
    f'https://api.hypeproxy.io/Utils/DirectRenewIp/{PROXY_ID}',
    headers=HEADERS
)

# Verify new IP
response = requests.get(
    f'https://api.hypeproxy.io/Utils/GetExternalIp/{PROXY_ID}',
    headers=HEADERS
)
print(f'New IP: {response.text}')

Web Scraping Example

A complete example that scrapes a website using a proxy with IP rotation:

import requests
import time

PROXY_HOST = 'proxy-host'
PROXY_PORT = 'port'
PROXY_USER = 'username'
PROXY_PASS = 'password'
API_TOKEN = 'YOUR_API_TOKEN'
PROXY_ID = 'YOUR_PROXY_ID'

proxies = {
    'http': f'http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}',
    'https': f'http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}',
}

headers = {'Authorization': f'Bearer {API_TOKEN}'}
urls = ['https://example.com/page/1', 'https://example.com/page/2']

for i, url in enumerate(urls):
    # Rotate IP every 50 requests
    if i > 0 and i % 50 == 0:
        requests.get(
            f'https://api.hypeproxy.io/Utils/DirectRenewIp/{PROXY_ID}',
            headers=headers
        )
        time.sleep(5)  # Wait for rotation

    response = requests.get(url, proxies=proxies)
    print(f'{url}: {response.status_code}')
    time.sleep(2)  # Polite delay between requests

Using with Selenium

For browser automation with Selenium:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument('--proxy-server=http://proxy-host:port')

driver = webdriver.Chrome(options=chrome_options)
driver.get('https://api.ipify.org')
print(f'Browser IP: {driver.page_source}')
driver.quit()

For authenticated proxies with Selenium, use a browser extension or a tool like selenium-wire.

Tips

• Use sessions: requests.Session() reuses connections for better performance.
• Handle errors: Wrap requests in try/except blocks and retry on failure.
• Respect robots.txt: Check the target site's robots.txt before scraping.
• Rotate user agents: Combine proxy rotation with user-agent rotation for better stealth.