# User Guide This page summarizes how to use the main features of pyinaturalist. ## Installation Installation instructions: :::{tab} Pip Install the latest stable version with pip: ``` pip install pyinaturalist ``` ::: :::{tab} Conda Or install from conda-forge, if you prefer: ``` conda install -c conda-forge pyinaturalist ``` ::: :::{tab} Pre-release If you would like to use the latest development (pre-release) version: ``` pip install --pre pyinaturalist ``` ::: :::{tab} Local development See {ref}`contributing` for details on setup for local development. ::: :::{admonition} Python version compatibility :class: toggle, tip pyinaturalist currently requires **python 3.6+**. If you need to use an older version of python, here are the last compatible versions of pyinaturalist: * **python 2.7:** pyinaturalist 0.1 * **python 3.4:** pyinaturalist 0.10 * **python 3.5:** pyinaturalist 0.11 * **python 3.6:** still supported, but expected to be dropped in a future release ::: ## Imports You can import all public functions and classes from the top-level `pyinaturalist` package: ``` >>> from pyinaturalist import Taxon, get_observations, pprint # etc. ``` Or you can just import everything: ``` >>> from pyinaturalist import * ``` ## Requests Requests generally follow the same format as the [API](https://api.inaturalist.org/v1) and [search URLs](https://forum.inaturalist.org/t/how-to-use-inaturalists-search-urls-wiki). For example, if you wanted to search observations by user, these three requests are equivalent: :::{tab} search URL ``` https://www.inaturalist.org/observations?user_id=tiwane,jdmore ``` ::: :::{tab} API request ``` https://api.inaturalist.org/v1/observations?user_id=tiwane%2Cjdmore ``` ::: :::{tab} pyinaturalist search ```python >>> get_observations(user_id=['tiwane', 'jdmore']) ``` ::: Compared to search URLs and raw API requests, pyinaturalist provides some conveniences for making requests easier: * Python lists instead of comma-separated strings * Python booleans instead of JS-style boolean strings or `1`/`0` * Python file-like objects or file paths for photo and sound uploads * Python {py:class}`~datetime.date` and {py:class}`~datetime.datetime` objects instead of date/time strings * Simplified data formats for `POST` and `PUT` requests * Simplified pagination * Validation for multiple-choice parameters And more, depending on the function. See the {ref}`reference-docs` section for a complete list of functions available. ## Responses API responses are returned as JSON, with some python type conversions applied (similar to the request type conversions mentioned above). Example response data is shown in the documentation for each request function, for example {py:func}`~pyinaturalist.v1.observations.get_observations`. ### API Data vs Web UI Here is how some of those response fields correspond to observation details shown on iNaturalist.org: ```{figure} images/inat-observation-page-annotated.png ``` And here is what that same observation looks like in JSON: :::{admonition} Observation response JSON :class: toggle ```{literalinclude} sample_data/get_observation_2.json ``` ::: ### Previewing Responses These responses can contain large amounts of response attributes, making it somewhat cumbersome if you just want to quickly preview results (for example, in a Jupyter notebook). For that purpose, the {py:func}`~pyinaturalist.formatters.pprint` function is included to format response data as a condensed, color-highlighted table. **Examples:** :::{tab} Observations ``` >>> from pyinaturalist import get_observations, pprint >>> observations = get_observations(user_id='niconoe', per_page=5) >>> pprint(observations) ID Taxon ID Taxon Observed on User Location ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82974075 61546 Species: Nemophora degeerella (Yellow-barred Longhorn) Jun 14, 2021 niconoe 1428 Braine-l'Alleud, Belgique 82827577 48201 Family: Scarabaeidae (Scarabs) Jun 13, 2021 niconoe 1428 Braine-l'Alleud, Belgique 82826778 48201 Family: Scarabaeidae (Scarabs) Jun 13, 2021 niconoe 1428 Braine-l'Alleud, Belgique 82696354 209660 Species: Chrysolina americana (Rosemary Beetle) Jun 12, 2021 niconoe 1420 Braine-l'Alleud, Belgique 82696334 472617 Species: Tomocerus vulgaris Jun 07, 2021 niconoe 1428 Braine-l'Alleud, Belgique ``` ::: :::{tab} Places ``` >>> from pyinaturalist import get_places, pprint >>> places = get_places_autocomplete('Vale') >>> pprint(places) ID Latitude Longitude Name Category URL ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96877 49.5189 -2.5190 Vale https://www.inaturalist.org/places/96877 21951 -16.8960 -40.8349 Fronteira dos Vales https://www.inaturalist.org/places/21951 23663 -6.3677 -41.8001 Valença do Piauí https://www.inaturalist.org/places/23663 24222 -27.2220 -53.6338 Pinheirinho do Vale https://www.inaturalist.org/places/24222 24374 -29.8309 -52.1121 Vale Verde https://www.inaturalist.org/places/24374 24442 -10.3841 -62.0939 Vale do Paraíso https://www.inaturalist.org/places/24442 103902 44.7355 27.5412 Valea Ciorii https://www.inaturalist.org/places/103902 103905 44.7529 26.8481 Valea Macrisului https://www.inaturalist.org/places/103905 105015 44.6805 24.0224 Valea Mare https://www.inaturalist.org/places/105015 104268 46.7917 27.0905 Valea Ursului https://www.inaturalist.org/places/104268 ``` ::: :::{tab} Places (with terminal colors) ```{figure} images/pprint_table.png ``` ::: ## Models Data models ({py:mod}`pyinaturalist.models`) are included for all API response types. These allow working with typed python objects instead of raw JSON. These are not used by default in the API query functions, but you can easily use them as follows: ```python >>> from pyinaturalist import Observation, get_observations >>> response = get_observations(user_id='my_username) >>> observations = Observation.from_json_list(response) ``` In a future release, these models will be fully integrated with the API query functions. ## Pagination Most endpoints support pagination, using the parameters: * `page`: Page number to get * `per_page`: Number of results to get per page * `count_only=True`: This is just a shortcut for `per_page=0`, which will return only the total number of results, not the results themselves. The default and maximum `per_page` values vary by endpoint, but it's 200 for most endpoints. To get all pages of results and combine them into a single response, use `page='all'`. Note that this replaces the `get_all_*()` functions from pyinaturalist\<=0.12. (auth)= ## Authentication For any endpoints that create, update, or delete data, you will need to authenticate using an OAuth2 access token. This requires both your iNaturalist username and password, and separate "application" credentials. :::{note} Read-only requests generally don't require authentication; however, if you want to access private data visible only to your user (for example, obscured or private coordinates), you will need to use an access token. ::: **Summary:** 1. Create an iNaturalist application 2. Use {py:func}`.get_access_token` with your user + application credentials to get an access token 3. Pass that access token to any API request function that uses it ### Creating an Application :::{admonition} Why do I need to create an application? :class: toggle, tip iNaturalist uses OAuth2, which provides several different methods (or "flows") to access the site. For example, on the [login page](https://www.inaturalist.org/login), you have the option of logging in with a username/password, or with an external provider (Google, Facebook, etc.): ```{image} images/inat-user-login.png :alt: Login form :width: 150 ``` Outside of iNaturalist.org, anything else that uses the API to create or modify data is considered an "application," even if you're just running some scripts on your own computer. See [iNaturalist documentation](https://www.inaturalist.org/pages/api+reference#auth) for more details on authentication. ::: First, go to [New Application](https://www.inaturalist.org/oauth/applications/new) and fill out the following pieces of information: * **Name:** Any name you want to come up with. For example, if this is associated with a GitHub repo, you can use your repo name. * **Description:** A brief description of what you'll be using this for. For example, *"Data access for my own observations"*. * **Confidential:** ✔️ This should be checked. * **URL and Redirect URI:** Just enter the URL to your GitHub repo, if you have one; otherwise any placeholder like "" will work. ```{image} images/inat-new-application.png :alt: New Application form :width: 300 ``` You should then see a screen like this, which will show your new application ID and secret. These will only be shown once, so save them somewhere secure, preferably in a password manager. ```{image} images/inat-new-application-complete.png :alt: Completed application form :width: 400 ``` ### Basic Usage There are a few different ways you can pass your credentials to iNaturalist. First, you can pass them as keyword arguments to {py:func}`.get_access_token`: ```python >>> from pyinaturalist import get_access_token >>> access_token = get_access_token( >>> username='my_inaturalist_username', # Username you use to login to iNaturalist.org >>> password='my_inaturalist_password', # Password you use to login to iNaturalist.org >>> app_id='33f27dc63bdf27f4ca6cd95dd', # OAuth2 application ID >>> app_secret='bbce628be722bfe2abde4', # OAuth2 application secret >>> ) ``` ### Environment Variables You can also provide credentials via environment variables instead of arguments. The environment variable names are the keyword arguments in uppercase, prefixed with `INAT_`: * `INAT_USERNAME` * `INAT_PASSWORD` * `INAT_APP_ID` * `INAT_APP_SECRET` **Examples:** :::{tab} Python ```python >>> import os >>> os.environ['INAT_USERNAME'] = 'my_inaturalist_username' >>> os.environ['INAT_PASSWORD'] = 'my_inaturalist_password' >>> os.environ['INAT_APP_ID'] = '33f27dc63bdf27f4ca6cd95df' >>> os.environ['INAT_APP_SECRET'] = 'bbce628be722bfe283de4' ``` ::: :::{tab} Unix (MacOS / Linux) ```bash export INAT_USERNAME="my_inaturalist_username" export INAT_PASSWORD="my_inaturalist_password" export INAT_APP_ID="33f27dc63bdf27f4ca6cd95df" export INAT_APP_SECRET="bbce628be722bfe283de4" ``` ::: :::{tab} Windows CMD ```bat set INAT_USERNAME="my_inaturalist_username" set INAT_PASSWORD="my_inaturalist_password" set INAT_APP_ID="33f27dc63bdf27f4ca6cd95df" set INAT_APP_SECRET="bbce628be722bfe283de4" ``` ::: :::{tab} PowerShell ```powershell $Env:INAT_USERNAME="my_inaturalist_username" $Env:INAT_PASSWORD="my_inaturalist_password" $Env:INAT_APP_ID="33f27dc63bdf27f4ca6cd95df" $Env:INAT_APP_SECRET="bbce628be722bfe283de4" ``` ::: Note that in any shell, these environment variables will only be set for your current shell session. I.e., you can't set them in one terminal and then access them in another. ### Keyring Integration To handle your credentials more securely, you can store them in your system keyring. You could manually store and retrieve them with a utility like [secret-tool](https://manpages.ubuntu.com/manpages/xenial/man1/secret-tool.1.html) and place them in environment variables as described above, but there is a much simpler option. Direct keyring integration is provided via [python keyring](https://github.com/jaraco/keyring). Most common keyring bakcends are supported, including: * macOS [Keychain](https://en.wikipedia.org/wiki/Keychain_%28software%29) * Freedesktop [Secret Service](http://standards.freedesktop.org/secret-service/) * KDE [KWallet](https://en.wikipedia.org/wiki/KWallet) * [Windows Credential Locker](https://docs.microsoft.com/en-us/windows/uwp/security/credential-locker) To store your credentials in the keyring, run {py:func}`.set_keyring_credentials`: ```python >>> from pyinaturalist.auth import set_keyring_credentials >>> set_keyring_credentials( >>> username='my_inaturalist_username', >>> password='my_inaturalist_password', >>> app_id='33f27dc63bdf27f4ca6cd95df', >>> app_secret='bbce628be722bfe283de4', >>> ) ``` Afterward, you can call {py:func}`.get_access_token` without any arguments, and your credentials will be retrieved from the keyring. You do not need to run {py:func}`.set_keyring_credentials` again unless you change your iNaturalist password. ### Password Manager Integration Keyring integration can be taken a step further by managing your keyring with a password manager. This has the advantage of keeping your credentials in one place that can be synced across multiple machines. [KeePassXC](https://keepassxc.org/) offers this feature for macOS and Linux systems. See this guide for setup info: [KeepassXC and secret service, a small walk-through](https://avaldes.co/2020/01/28/secret-service-keepassxc.html). ```{figure} images/password_manager_keying.png Credentials storage with keyring + KeePassXC ``` ## Dry-run mode While developing and testing, it can be useful to temporarily mock out HTTP requests, especially requests that add, modify, or delete real data. Pyinaturalist has some settings to make this easier. ### Dry-run all requests To enable dry-run mode, set the `DRY_RUN_ENABLED` variable. When set, requests will not be sent but will be logged instead: ```python >>> import logging >>> import pyinaturalist >>> >>> # Enable at least INFO-level logging >>> logging.basicConfig(level='INFO') >>> >>> pyinaturalist.DRY_RUN_ENABLED = True >>> get_taxa(q='warbler', locale=1) {'results': \[\], 'total_results': 0} INFO:pyinaturalist.api_requests:Request: GET, https://api.inaturalist.org/v1/taxa, params={'q': 'warbler', 'locale': 1}, headers={'Accept': 'application/json', 'User-Agent': 'Pyinaturalist/0.9.1'} ``` You can also set this as an environment variable (case-insensitive): ```bash $ export DRY_RUN_ENABLED=true $ python my_script.py ``` ### Dry-run only write requests If you would like to send real `GET` requests but mock out any requests that modify data (`POST`, `PUT`, `DELETE`, etc.), you can use the `DRY_RUN_WRITE_ONLY` variable instead: ```python >>> pyinaturalist.DRY_RUN_WRITE_ONLY = True >>> # Also works as an environment variable >>> import os >>> os.environ\["DRY_RUN_WRITE_ONLY"\] = 'True' ``` ## User Agent While not mandatory, it's good practice to include a [user-agent](https://en.wikipedia.org/wiki/User_agent) in your API calls. This field can be either something that identifies the project or its contact person. You can set this globally: ```python >>> import pyinaturalist >>> >>> pyinaturalist.user_agent = "MyCoolAndroidApp/2.0 (using Pyinaturalist)" >>> # From now on, all API calls will use this user-agent. ``` Or to set this for individual requests, all API functions accept an optional `user_agent` parameter: ```python >>> from pyinaturalist import get_observation >>> get_observation(observation_id=16227955, user_agent='Jane Doe \') ``` If not configured, `Pyinaturalist/` will be used. ## API Recommended Practices See [API Recommended Practices](https://www.inaturalist.org/pages/api+recommended+practices) on iNaturalist for more general usage information and notes.