Cannot install 1.5.0 on Python 3.9

[nutjob4life]

Checked for duplicates

Yes - I've already checked

🐛 Describe the bug

When I tried to install pds.api-client==1.5.0 from test.pypi.org, I encountered an error:

bash-3.2$ date -u
Wed Feb 14 15:59:07 UTC 2024
bash-3.2$ python3.9 -m venv venv
bash-3.2$ cd venv
bash-3.2$ . bin/activate
(venv) bash-3.2$ pip install --quiet --upgrade setuptools wheel build pip
(venv) bash-3.2$ pip install --index-url https://test.pypi.org/simple/ pds.api-client==1.5.0
Looking in indexes: https://test.pypi.org/simple/
Collecting pds.api-client==1.5.0
  Using cached https://test-files.pythonhosted.org/packages/e3/7d/e830c1927b828306f6cc731e1c4f753101e8392c597b74ef59e458a31d45/pds.api_client-1.5.0-py3-none-any.whl.metadata (842 bytes)
INFO: pip is looking at multiple versions of pds-api-client to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement urllib3<2.1.0,>=1.25.3 (from pds-api-client) (from versions: 1.23)
ERROR: No matching distribution found for urllib3<2.1.0,>=1.25.3
(venv) bash-3.2$ 

🕵️ Expected behavior

Installation

📜 To Reproduce

1. Create a virtual environment with Python 3.9: python3.9 -m venv venv
2. Go into that directory and activate the environment: cd venv; . bin/activate
3. Install pds.api-client 1.5.0 from test.pypi.org: pip install --index-url https://test.pypi.org/simple/ pds.api-client==1.5.0

🖥 Environment Info

• macOS 14.2.1
• Python 3.9.18

📚 Version of Software Used

1.5.0 from test.pypi.org

🩺 Test Data / Additional context

Currently blocks NASA-PDS/deep-archive#159

[tloubrieu-jpl]

Hi @nutjob4life ,

it is a weird thing, it sounds like the pip is getting also the dependencies from test pypi and they are missing there. To get the dependencies from the official pypi, you need to run pip install as follow:

pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pds.api-client==1.5.0

[nutjob4life]

Ah, I never heard of --extra-index-url before.

Looks like that maps to dependency_links in setup.cfg. That means for this patch, it'll be coded specifically for test.pypi.org with dependency_links set to the real pypi.org. Would it be better to wait until pds.api-client v1.5.0 is live on the real pypi.org so we don't have to then have a second patch to deep-archive that removes those extra lines?

[tloubrieu-jpl]

Hi @nutjob4life ,

Yes, I fixed the second thing I wanted to in pds.api-client so I will be able to release it later today or tomorrow. I sent you the link earlier on testpypi so that you can work on it as soon as possible but did not realize that it was easier for you to make it all at once.

I'll keep you informed when it is published to unblock this ticket.

[nutjob4life]

@tloubrieu-jpl thanks so much! Yeah I think it'd be best to use the "official" release at this point. I appreciate it! 🕺 💃

[tloubrieu-jpl]

The official release is now on pypi https://pypi.org/project/pds.api-client/

It is a chicken and egg thing, since the API service is not in deployed in production yet. I'll put it in product at the same time when the new version of deeep-archive is going to be released.

[tloubrieu-jpl]

@nutjob4life 🖐️

[nutjob4life]

It is a chicken and egg thing

Hence why it's important for both servers and clients to use version pins 😄

[nutjob4life]

@tloubrieu-jpl getting closer! 😀

Here's where I get stuck now:

$ date -u
Thu Feb 15 19:32:23 UTC 2024
$ python3.9 -m venv venv
$ . venv/bin/activate
$ pip install --quiet --upgrade pip setuptools wheel build
$ pip install --quiet pds.api-client==1.5.0
$ python3
Python 3.9.18 (main, Aug 24 2023, 18:16:58) 
[Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pds.api_client.model.pds_product import PdsProduct
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/venv/lib/python3.9/site-packages/pds/api_client/model/pds_product.py", line 22, in <module>
    import frozendict  # noqa: F401
ModuleNotFoundError: No module named 'frozendict'

Is there a chance we're missing a dependency in the new client?

[jordanpadams]

@tloubrieu-jpl ☝️

[tloubrieu-jpl]

Hi @nutjob4life ,

I am having a different error, but still an error, I am investigating that now:

 % python3.9 
Python 3.9.7 (v3.9.7:1016ef3790, Aug 30 2021, 16:39:15) 
[Clang 6.0 (clang-600.0.57)] on darwin  python3.10          python3.11          python3.11-intel64  python3.12-config   python3.8-config    python3.9-config    
Type "help", "copyright", "credits" or "license" for more information.
>>> from pds.api_client.model.pds_product import PdsProduct
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pds.api_client.model'

[tloubrieu-jpl]

Hi @nutjob4life ,

Can you try from pds.api_client.models.pds_product import PdsProduct with an s at the end of model. I am not sure when this was changed.

Thanks,

Thomas

[nutjob4life]

@tloubrieu-jpl getting closer! Can you tell me what I'm doing wrong now?

$ date
Tue Feb 20 12:51:29 CST 2024
$ python3.9 -m venv venv
$ . venv/bin/activate
(venv) $ pip install --quiet --upgrade pip
(venv) $ pip install --quiet pds.api-client==1.5.0
(venv) $ python3
Python 3.9.18 (main, Aug 24 2023, 18:16:58) 
[Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pds.api_client import Configuration, ApiClient
>>> from pds.api_client.api.bundles_api import BundlesApi
>>> 
>>> config = Configuration()
>>> config.host = 'https://pds.nasa.gov/api/search-en-gamma/1/'
>>> client = ApiClient(config)
>>> bundles = BundlesApi(client)
>>> print(bundles.bundle_by_lidvid('urn:nasa:pds:magellan_gxdr::1.0'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/venv/lib/python3.9/site-packages/pds/api_client/api/bundles_api.py", line 62, in bundle_by_lidvid
    return self.bundle_by_lidvid_with_http_info(identifier, **kwargs)  # noqa: E501
  File "/private/tmp/venv/lib/python3.9/site-packages/pds/api_client/api/bundles_api.py", line 136, in bundle_by_lidvid_with_http_info
    return self.api_client.call_api(
TypeError: call_api() got multiple values for argument 'body'
>>> 

[jordanpadams]

@nutjob4life is there are reason we are using the gamma API?

config.host = 'https://pds.nasa.gov/api/search/1/'

Also, I think we don't want to use the BundlesApi if we can avoid it. It may not be as backwards compatible as we thought. Probably ProductsApi is more robust and then using the members-of to find it's children.

Here are some examples: Bundle information: https://pds.nasa.gov/api/search/1/products/urn:nasa:pds:mars2020_sherloc::1.0

Members of a bundle (collections): https://pds.nasa.gov/api/search/1/products/urn:nasa:pds:mars2020_sherloc::1.0/members

Members of a collection (individual products): https://pds.nasa.gov/api/search/1/products/urn:nasa:pds:mars2020_mission:document_camera::1.1/members

Note: Sorry I don't know what the API classes/methods are associated with this functionality.

[nutjob4life]

@jordanpadams wrote:

is there are reason we are using the gamma API?

Because @tloubrieu-jpl said that's where the new API is running 😇 See this issue.

As for using the BundlesApi, that's what the Deep Archive has historically used in order to retrieve the title (used in the label for the submission information package) and the label's URL (so we know how many characters to strip off the checksum manifest's entries).

[jordanpadams]

@nutjob4life copy that. you can ignore my previous comments. The other API endpoints will bring back that content as well.

[tloubrieu-jpl]

Hi @nutjob4life ,

I was able to get the bundle title with the following code:

from pds.api_client import Configuration
from pds.api_client import ApiClient
from pds.api_client.api.all_products_api import AllProductsApi

configuration = Configuration()
configuration.host = 'https://pds.nasa.gov/api/search-en-gamma/1'
api_client = ApiClient(configuration)
all_products = AllProductsApi(api_client)
exact_bundle = all_products.select_by_lidvid('urn:nasa:pds:magellan_gxdr::1.0')
print(exact_bundle.title)

Did the code you have work on the production deployment, with the previous release of pds.api-client ? I don't think this can work anymore anyway with the next release now deployed and gamma and pds.api-client v1.5.0.

Sorry the error message is not more explicit but the line from pds.api_client.api.bundles_api import BundlesApi should not work from the beginning since this package/class is not provided anymore by the pds.api-client.

[tloubrieu-jpl]

To be more clear the code you have cannot work with the latest version of the pds.api-client because this class does not exist anymore as well as the API back-end URLs.

[nutjob4life]

Okay I think we've established that this was a wild goose chase and we can close this issue! 👏