[nssdca-delivery] PDS_ATM Pack9v2 (resubmission)

[tinagueth]

Discipline Node Information

• Delivering Node: PDS_ATM

• NSSDCA Delivery Package: pdsatm_pack9v2_20230525.tar.gz

• Validation report: pdsatm_pack9v2_validationreports.tar.gz

NOTE: If you have multiple delivery packages, we strongly encourage you to submit these in batches of 3 to 10 per issue with one ZIP file of the packages and another ZIP file of the validation reports. Please use a descriptive title, such as "Node Mission misc batch #".

---

Engineering Node Process

See the internal EN process at https://pds-engineering.jpl.nasa.gov/content/nssdca_interface_process

[c-suh]

@tinagueth thank you for the resubmission! Unfortunately, I am still seeing the same errors. In an effort to troubleshoot, would you mind letting me know which version of Deep Archive you used for this resubmission?

[tinagueth]

@c-suh I have not updated the deep-archive since installing it last year. Is there an update? I checked an it says pds-deep-archve 1.0.0 is that the issue or is it still the validation part (Lyle does those).

[c-suh]

@tinagueth apologies for the delay. There are later releases of the pds-deep-archive; however, I think the fix you need is in the pre-release, which we typically do not ask our users to use. I will confirm and get back to you.

[c-suh]

@tinagueth thank you for your patience! Please update your Deep Archive to v1.1.3 and use that before resubmitting your package in this issue (in a comment or by editing the main description - either is fine). Thank you!

[tinagueth]

@c-suh sorry for the delayed reply. I ran the following commands to update my deep-archive version source $HOME/.virtualenvs/pds-deep-archive/bin/activate pip install pds.deeparchive --upgrade which then required me to update the pip version as mine was version 9 or such. So I ran command pip install --upgrade pip

When I check with pip3 --version, it shows pip 21.3.1 when I check pds-deep-archive --version, it shows pds-deep-archive 1.0.0

did I do something wrong? I logged out and back in, but it still shows the same version info. I re-did the installation commands for both and it stated that the files are up to date (or something along those lines). I do not want to rerun the deep-archive for the package until I know this is normal. I hope this makes sense. Thank you.

[c-suh]

@tinagueth after the pip upgrade and attempt to upgrade pds.deeparchive, did the output reflect a successful upgrade or did it note any errors, such as the following?

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> lxml

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[tinagueth]

@c-suh here is what I get when I run the deep-archive update and check the version info (along with the pip version info). I removed some info but kept the square brackets in...

[ ]$ source $HOME/.virtualenvs/pds-deep-archive/bin/activate (pds-deep-archive) [ ]$ pip install pds.deeparchive --upgrade Requirement already satisfied: pds.deeparchive in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (1.0.0) Requirement already satisfied: zope.interface in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.deeparchive) (5.5.2) Requirement already satisfied: lxml in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.deeparchive) (4.9.3) Requirement already satisfied: zope.component in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.deeparchive) (5.1.0) Requirement already satisfied: pds.api-client==0.4.0 in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.deeparchive) (0.4.0) Requirement already satisfied: setuptools in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.deeparchive) (59.6.0) Requirement already satisfied: certifi in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.api-client==0.4.0->pds.deeparchive) (2023.5.7) Requirement already satisfied: python-dateutil in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.api-client==0.4.0->pds.deeparchive) (2.8.2) Requirement already satisfied: six>=1.10 in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.api-client==0.4.0->pds.deeparchive) (1.16.0) Requirement already satisfied: urllib3>=1.15 in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from pds.api-client==0.4.0->pds.deeparchive) (1.26.16) Requirement already satisfied: zope.event in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from zope.component->pds.deeparchive) (4.6) Requirement already satisfied: zope.hookable>=4.2.0 in ./.virtualenvs/pds-deep-archive/lib/python3.6/site-packages (from zope.component->pds.deeparchive) (5.4) (pds-deep-archive) [ ]$ pds-deep-archive --version pds-deep-archive 1.0.0 (pds-deep-archive) [ ]$ pip3 --version pip 21.3.1 from .../.virtualenvs/pds-deep-archive/lib64/python3.6/site-packages/pip (python 3.6)

not sure if that helps, but I do not get the error message you provided. Thanks!

[jordanpadams]

@tinagueth looks like you are running Python 3.6, but PDS Deep Archive requires Python 3.9 or 3.10. It looks like you may have been lucky running on 3.6 with the previous version :-) . I would clean out that initial venv ($HOME/.virtualenvs/pds-deep-archive), install Python 3.9/3.10, and try to re-install in a new virtual environment.

[tinagueth]

@c-suh @jordanpadams I assumed that it might be the python version at this point. I have to check who has to update the virtual environment to a newer python version as I am not able to that part (no admin privileges for me. I will have to check with Lyle most likely on that). After the python3 version is updated, I am not sure what you man by cleaning out the initial venv... is that I file or directory that I need to update/remove prior to running the above installation commands for pds-deep-archive and pip)? Thanks for the quick reply

[jordanpadams]

@tinagueth if you would like to install Python 3.9 locally, you can try using Miniconda. I have done this in the past, and it works pretty well.

[tinagueth]

@jordanpadams I found out from Lyle that python 3.9 has been installed where I run deep-archive, however the older version (i.e. python 3.6.8) is being kept working as well due others requiring it currently. Lyle mentioned that I would need to specific python directly as python3.9 (current version is 3.9.16 when I check using: python3.9 --version). Is there a way of defining the required python version when running the pds-deep-archive command? I still use the Example 2 way as given on your website (https://nasa-pds.github.io/deep-archive/usage/index.html#example-2-bundle-referencing-collections-by-lid). I should also say thank you for having examples and explanations for this as it made it easier for me to learn when I took over :)

[jordanpadams]

@tinagueth you will most likely need to create a new virtual environment for this using Python 3.9. So I would run the instructions from the Installation page, but you just replace any 3 with 3.9:

Remove existing virtual environment:

rm -fr $HOME/.virtualenvs/pds-deep-archive

Re-install with Python3.9

python3.9 -m venv $HOME/.virtualenvs/pds-deep-archive
source $HOME/.virtualenvs/pds-deep-archive/bin/activate
pip3.9 install pds.deeparchive

And then you should be good to go. If the python3.9 throws an error, you may need to run locate python3.9 to add that to your $PATH.

[tinagueth]

@jordanpadams Thanks for the detailed information. I did all the steps and even updated pip from 22.0.4 to 23.2. I also checked that python --version and got 3.9.16. However, I get the following error message when I compile the normal pds-deep-archive command for a dataset

$ .virtualenvs/pds-deep-archive/bin/pds-deep-archive -s PDS_ATM -b ... Traceback (most recent call last): File "/home/tgueth/.virtualenvs/pds-deep-archive/bin/pds-deep-archive", line 5, in from pds2.aipgen.main import main File "/home/tgueth/.virtualenvs/pds-deep-archive/lib/python3.9/site-packages/pds2/aipgen/main.py", line 35, in import sqlite3 File "/usr/bin/lib/python3.9/sqlite3/init.py", line 57, in from sqlite3.dbapi2 import File "/usr/bin/lib/python3.9/sqlite3/dbapi2.py", line 27, in from _sqlite3 import ModuleNotFoundError: No module named '_sqlite3'

Is this something that I have install or is it something that needs to be done by ICT? Thanks for all the help and information.

[jordanpadams]

@tinagueth make sure you source the virtual environment before execution:

$ source $HOME/.virtualenvs/pds-deep-archive/bin/activate
$ pds-deep-archive -s PDS_ATM -b ...

From the log messages, you can see it is looking in /usr/bin/lib/python3.9/ for /usr/bin/lib/python3.9/sqlite3/init.py, but it should be looking in /home/tgueth/.virtualenvs/pds-deep-archive/lib/python3.9. Additionally, pds-deep-archive should just be runnable as a global command once you source that virtual environment.

[c-suh]

Hi @tinagueth how has the Deep Archive installation gone? I am going to close this ticket, but feel free to message or email us for more information and to open a new ticket with the (re)corrected packages once they are ready. Thank you!

[tinagueth]

@c-suh I will resubmit the cases you closed (after re-running the updated deep-archive commands), along with other newer bundles. However, we are still waiting on getting our new computer and therefore the updated deep-archive installation fully ready. Almost there, so fingers crossed. Thank you :)