search

NASA-PDS / registry

PDS Registry provides service and software application necessary for tracking, searching, auditing, locating, and maintaining artifacts within the system. These artifacts can range from data files and label files, schemas, dictionary definitions for objects and elements, services, etc.
https://nasa-pds.github.io/registry
Apache License 2.0
3 stars 2 forks source link

Titan Treks PDS4 Utility Script #290

Closed mattanikiej closed 4 months ago

mattanikiej commented 4 months ago

🗒️ Summary

A utility script has been added that creates Product_Service pds4 xml files for Titan Treks OGC WMTS layers. These layers are scraped from the Titan Treks API, and the pds4 xml is populated with their information for each layer. After it builds out the files, it also creates a .tab inventory file for the Titan Treks API Product_Collection pds4 file.

⚙️ Test Data and/or Report

The files were tested by making sure the collection is able to be uploaded to a local registry, and all collection members appear when queried for products.

♻️ Related Issues

Resolves #284 Resolves #285

-->

mattanikiej commented 4 months ago

Also I don't see the collection XML. I was thinking you should add it along-side your script in the same directory and add a code which copies it from there to the destination_directory. I am suggesting that to make your package complete, self-contained, without a need to have a additional documentation somewhere else which has to explain that a collection PDS4 is needed and where to get it and where to push it...

Replying to this so that I can say it has been resolved

tloubrieu-jpl commented 4 months ago

Hi @mattanikiej ,

I tried the script following the manual on the README but I am getting an error:

% create-titan-treks-pds4 -d ~/tmp/titan
Traceback (most recent call last):
  File "/Users/loubrieu/git/registry/venv/bin/create-titan-treks-pds4", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/create_titan_treks_pds4.py", line 77, in main
    _, lidvid = psb.create_pds4_xml(data=data, save_xml=save_xml, dest=dest, verbose=verbose)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 27, in create_pds4_xml
    observation_area = create_observation_area(data, verbose)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 118, in create_observation_area
    observation_area.append(create_time_coordinates(data, verbose))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 179, in create_time_coordinates
    soup = BeautifulSoup(response.content, features='xml')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/venv/lib/python3.11/site-packages/bs4/__init__.py", line 250, in __init__
    raise FeatureNotFound(
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?

Do you think a dependency is missing in the setup.cfg ?

mattanikiej commented 4 months ago

Hi @mattanikiej ,

I tried the script following the manual on the README but I am getting an error:

% create-titan-treks-pds4 -d ~/tmp/titan
Traceback (most recent call last):
  File "/Users/loubrieu/git/registry/venv/bin/create-titan-treks-pds4", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/create_titan_treks_pds4.py", line 77, in main
    _, lidvid = psb.create_pds4_xml(data=data, save_xml=save_xml, dest=dest, verbose=verbose)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 27, in create_pds4_xml
    observation_area = create_observation_area(data, verbose)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 118, in create_observation_area
    observation_area.append(create_time_coordinates(data, verbose))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 179, in create_time_coordinates
    soup = BeautifulSoup(response.content, features='xml')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/venv/lib/python3.11/site-packages/bs4/__init__.py", line 250, in __init__
    raise FeatureNotFound(
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?

Do you think a dependency is missing in the setup.cfg ?

That's very interesting, it should be the beautifulsoup4 package. I used it since it makes parsing xml very easy, but it seems I need to also install lxml to use the xml parser after reading online. That sounds like dependency hell for such a small part though... I'll refactor when I get to work tomorrow using just the python xml parser.

mattanikiej commented 4 months ago

Hi @mattanikiej ,

I tried the script following the manual on the README but I am getting an error:

% create-titan-treks-pds4 -d ~/tmp/titan
Traceback (most recent call last):
  File "/Users/loubrieu/git/registry/venv/bin/create-titan-treks-pds4", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/create_titan_treks_pds4.py", line 77, in main
    _, lidvid = psb.create_pds4_xml(data=data, save_xml=save_xml, dest=dest, verbose=verbose)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 27, in create_pds4_xml
    observation_area = create_observation_area(data, verbose)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 118, in create_observation_area
    observation_area.append(create_time_coordinates(data, verbose))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/src/pds/registry/utils/titan_treks/product_service_builder.py", line 179, in create_time_coordinates
    soup = BeautifulSoup(response.content, features='xml')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loubrieu/git/registry/venv/lib/python3.11/site-packages/bs4/__init__.py", line 250, in __init__
    raise FeatureNotFound(
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?

Do you think a dependency is missing in the setup.cfg ?

Just changed the code so it doesn't use the beautiful soup and lxml dependency. Let me know if the command works now.