Session¶
Summary¶
Classes
|
Custom session class used for sending API requests. |
|
Bucket backed by a SQLite database and file lock. |
|
A mock response to return in dry-run mode. |
Functions
Clear all cached API responses |
|
|
Wrapper around |
|
Translate an environment variable to a boolean value, accounting for minor variations (case, None vs. |
|
Wrapper around |
|
Get a thread-local Session object with default settings. |
|
In some cases, we need to be sure we have the most recent version of a resource, for example when updating projects. |
|
A wrapper to determine if dry-run (aka test mode) has been enabled via either a constant or an environment variable. |
|
Wrapper around |
|
Wrapper around |
Module Contents¶
Session class and related functions for preparing and sending API requests
- class pyinaturalist.session.ClientSession(cache_file='/home/docs/.local/share/pyinaturalist/api_requests.db', cache_control=True, expire_after=None, urls_expire_after=None, per_second=1, per_minute=60, per_day=10000, burst=5, bucket_class=<class 'pyrate_limiter.sqlite_bucket.SQLiteBucket'>, backoff_factor=0.5, max_retries=5, timeout=20, user_agent=None, **kwargs)¶
Bases:
CacheMixin
,LimiterMixin
,Session
Custom session class used for sending API requests. Combines the following features and settings:
Caching
Rate-limiting (skipped for cached requests)
Retries
Timeouts
- __init__(cache_file='/home/docs/.local/share/pyinaturalist/api_requests.db', cache_control=True, expire_after=None, urls_expire_after=None, per_second=1, per_minute=60, per_day=10000, burst=5, bucket_class=<class 'pyrate_limiter.sqlite_bucket.SQLiteBucket'>, backoff_factor=0.5, max_retries=5, timeout=20, user_agent=None, **kwargs)¶
Get a Session object, optionally with custom settings for caching and rate-limiting.
- Parameters:
cache_file (
Union
[BinaryIO
,Path
,str
]) – Cache file path to use; defaults to the system default cache directorycache_control (
bool
) – Use server-provided Cache-Control headers to set cache expiration when possible (instead ofexpire_after
orurls_expire_after
)expire_after (
Union
[None
,int
,float
,str
,datetime
,timedelta
]) – How long to keep cached API requests; for advanced options, see requests-cache: ExpirationSee (urls_expire_after Glob patterns for per-URL cache expiration;) – requests-cache: URL Patterns
per_second (
float
) – Max requests per secondper_minute (
float
) – Max requests per minuteper_day (
float
) – Max requests per dayburst (
int
) – Max number of consecutive requests allowed before applying per-second rate-limitingbucket_class (
Type
[AbstractBucket
]) – Rate-limiting backend to use. Defaults to a persistent SQLite database.backoff_factor (
float
) – Factor for increasing delays between retriesmax_retries (
int
) – Maximum number of times to retry a failed requesttimeout (
int
) – Maximum number of seconds to wait for a response from the serveruser_agent (
Optional
[str
]) – Additional User-Agent info to pass to API requestskwargs – Additional keyword arguments for
CachedSession
and/orLimiterSession
- prepare_inat_request(method, url, access_token=None, files=None, headers=None, ids=None, json=None, params=None, allow_str_ids=False, **kwargs)¶
Translate pyinaturalist-specific options into standard request arguments
- Return type:
- request(method, url, headers=None, json=None, access_token=None, allow_redirects=False, allow_str_ids=False, dry_run=False, expire_after=None, files=None, ids=None, only_if_cached=False, raise_for_status=True, refresh=False, stream=False, timeout=None, verify=True, **params)¶
Wrapper around
requests.request()
with additional options specific to iNat API requests- Parameters:
method (
str
) – HTTP methodurl (
str
) – Request URLaccess_token (
Optional
[str
]) – access_token: the access token, as returned byget_access_token()
dry_run (
bool
) – Just log the request instead of sending a real requestexpire_after (
Union
[None
,int
,float
,str
,datetime
,timedelta
]) – How long to keep cached API requestsfiles (
Union
[BinaryIO
,Path
,str
,None
]) – File object, path, or URL to uploadids (
Union
[int
,Iterable
[int
],None
]) – One or more integer IDs used as REST resource(s) to requestonly_if_cached (
bool
) – Only return a response if it is cachedraise_for_status (
bool
) – Raise an exception if the response status is not 2xxrefresh (
bool
) – Skip reading from the cache and always fetch a fresh responsestream (
bool
) – Stream the response contenttimeout (
Optional
[int
]) – Time (in seconds) to wait for a response from the server; if exceeded, arequests.exceptions.Timeout
will be raised.verify (
bool
) – Verify SSL certificatesparams (
Dict
[str
,Any
]) – All other keyword arguments will be interpreted as request parameters
- Return type:
- Returns:
API response
- send(request, dry_run=False, expire_after=None, refresh=False, retries=None, timeout=None, **kwargs)¶
Send a request with caching, rate-limiting, and retries
- Parameters:
request (
Union
[Request
,PreparedRequest
,CachedRequest
]) – Prepared request to senddry_run (
bool
) – Just log the request instead of sending a real requestexpire_after (
Union
[None
,int
,float
,str
,datetime
,timedelta
]) – How long to keep cached API requestsrefresh (
bool
) – Skip reading from the cache and always fetch a fresh responsetimeout (
Optional
[int
]) – Maximum number of seconds to wait for a response from the server
Note:
requests.Session.send()
accepts separate timeout values for connect and read timeouts. Thetimeout
argument will be used as the read timeout.- Return type:
- class pyinaturalist.session.FileLockSQLiteBucket(lock_path=PosixPath('/tmp/pyrate_limiter.lock'), **kwargs)¶
Bases:
FileLockSQLiteBucket
Bucket backed by a SQLite database and file lock. Suitable for usage from multiple processes with no shared state. Requires installing [py-filelock](https://py-filelock.readthedocs.io).
The file lock is reentrant and shared across buckets, allowing a process to access multiple buckets at once.
This modified class allows setting a custom lock path.
- class pyinaturalist.session.MockResponse(request=None, **kwargs)¶
Bases:
CachedResponse
A mock response to return in dry-run mode. This behaves the same as a cached response, but with the following additions:
Adds default response values
Returns a
defaultdict
when callingjson()
, to accommodate checking for arbitrary keys
- json(**kwargs)¶
Returns the json-encoded content of a response, if any.
- Parameters:
**kwargs – Optional arguments that
json.loads
takes.- Raises:
requests.exceptions.JSONDecodeError – If the response body does not contain valid json.
- pyinaturalist.session.clear_cache()¶
Clear all cached API responses
- pyinaturalist.session.delete(url, session=None, **kwargs)¶
Wrapper around
requests.delete()
with additional options specific to iNat API requests- Return type:
- pyinaturalist.session.env_to_bool(environment_variable)¶
Translate an environment variable to a boolean value, accounting for minor variations (case, None vs. False, etc.)
- Return type:
- pyinaturalist.session.get(url, session=None, **kwargs)¶
Wrapper around
requests.get()
with additional options specific to iNat API requests- Return type:
- pyinaturalist.session.get_local_session(**kwargs)¶
Get a thread-local Session object with default settings. This will be reused across requests to take advantage of connection pooling and (optionally) caching. If used in a multi-threaded context (for example, a
ThreadPoolExecutor
), this will create and store a separate session object for each thread.- Parameters:
kwargs – Keyword arguments for
ClientSession()
- Return type:
- pyinaturalist.session.get_refresh_params(endpoint)¶
In some cases, we need to be sure we have the most recent version of a resource, for example when updating projects. Normally we would handle this with cache headers, but the CDN cache does not respect these if the cached response is less than ~2 minutes old.
The iNat webapp handles this by adding an extra v request parameter, which results in a different cache key. This function does the same by tracking a separate rate limit per value, and finding the lowest value that is certain to result in a fresh response.
- Return type:
- pyinaturalist.session.is_dry_run_enabled(method)¶
A wrapper to determine if dry-run (aka test mode) has been enabled via either a constant or an environment variable. Dry-run mode may be enabled for either write requests, or all requests.
- Return type:
- pyinaturalist.session.post(url, session=None, **kwargs)¶
Wrapper around
requests.post()
with additional options specific to iNat API requests- Return type:
- pyinaturalist.session.put(url, session=None, **kwargs)¶
Wrapper around
requests.put()
with additional options specific to iNat API requests- Return type: