Using Python requests

Posted on November 17, 2022

The Python requests module is an elegant and simple HTTP library for making requests in Python. Requests allows you to send HTTP requests extremely easily. No need to manually add query strings to URLs, or form-encode your POST data. Keep-alive and HTTP connection pooling are 100% automatic, thanks to urllib3.

With requests, you can focus on interacting with APIs and web services without having to worry about all the fiddly underlying details. In this post, we’ll go over some of the basics of using requests to help you get started.

Installing Requests

Installing requests is simple with pip:

pip install requests

This will install the latest version of requests from PyPI.

Making Requests

The requests module allows you to make HTTP requests extremely easily. Let’s look at a simple GET request:

import requests

response = requests.get('https://api.example.com/data')

To make a POST request, just pass the data parameter:

requests.post('https://api.example.com/user', data={'name':'John'})

You can also pass query string parameters as a dictionary to the params argument:

requests.get('https://api.example.com/search', params={'query':'requests'}) 

For POST requests, you may want to send data in JSON format. Set the json parameter to a dictionary:

data = {'name':'John', 'age':22}
requests.post('https://api.example.com/user', json=data)

There are convenience methods like put(), delete(), head() and options() too for other HTTP verbs.

Response Content

The Response object returned by requests contains all the information about the request.

The status code is accessed through the status_code attribute:

response = requests.get('https://api.example.com/data')
print(response.status_code)

The text content is accessible by calling .text on the Response:

print(response.text)

For JSON data, use the json() method:

data = response.json()
print(data['name'])  

If any errors occur, you can check the Response error attribute. response.raise_for_status() will raise an exception if the status is 4XX or 5XX:

response.raise_for_status()
response.status_code == requests.codes.ok

Advanced Usage

Requests makes it easy to do more advanced operations like persisting sessions across requests using Sessions:

session = requests.Session()

session.get('https://api.example.com/user')

r = session.post('https://api.example.com/login', data={'user':'foo', 'pass':'bar'})

You can stream responses directly without having to load everything into memory:

with requests.get('https://api.example.com/bigfile', stream=True) as r:
    for chunk in r.iter_content(1024):
        print(chunk)

To use HTTP Basic Auth or Digest Auth, you can pass a tuple of (username, password) to the auth parameter:

requests.get('https://api.example.com/', auth=('user','pass'))

Requests supports proxies, timeout settings, SSL verification, and much more. See the full documentation for details.

Recap

Hopefully this gives you a quick overview of how to get started with the Python requests module! Here’s a quick recap:

  • Installing is easy with pip
  • Make HTTP requests with GET, POST, PUT, DELETE and more
  • Access response status, text, headers, JSON
  • Use advanced features like sessions, proxies, authentication

With its elegant API, Requests makes working with HTTP and APIs in Python a breeze. Be sure to check out the full documentation for more details. Happy making requests!

The requests module is a crucial part of the Python ecosystem. By handling the nitty-gritty details of making HTTP requests, it allows developers to focus on the essentials of their application. Requests makes integrating with web services and APIs a straightforward process.

I encourage you to try it out on your next Python project that involves any type of HTTP communication. The simple yet powerful API will quickly become something you can’t live without!

python programming devops