Publish Directory - Githubissues

woodRock / psychic-invention

NZODN Data Ingestion Project

0 stars 0 forks source link

Publish Directory #8

Closed woodRock closed 3 years ago

woodRock commented 3 years ago

Goal

Gain access to the /data/niwa/publish directory to copy the zipfile of the dataset to. Move the zip there.

Tasklist

[x] Raise ticket to Service Desk
[x] Gain access to the directory
[x] Email Glenn to clarify process
[x] Create bio sub-directory
[x] Copy temporary zipfile to bio directory

Success Criteria

I have access to the directory where the other zipfiles are stored. And have copied the temporary zipfile into its own folder called bio.

woodRock commented 3 years ago

The publish folder is owned by the robot user. This suggests, and IT confirmed, that this is a dedicated service user for uploading data. We need to gain access to this robot user in order to publish the zipfile for the biological dataset.

woodRock commented 3 years ago

Suzi has suggested we create a subdirectory called bio or benthic (Andrea suggests consistency for Postgres schema name as well #5) within the publish directory.

woodRock commented 3 years ago

The service desk will provide sudo access to the robot user. With this permission we will be able to write files to the relevant directory. This removes the barrier that was impeding progress. However, it may be wise to follow up with Glenn, to see if this was the same process that he used. As IT see many red flags with this approach.

woodRock commented 3 years ago

woodRock commented 3 years ago

We created a directory called bio within the publish directory. Also for reproducibility, a script was made to publish a zipfile to that same directory.

woodRock commented 3 years ago

The zip file can now be downloaded from this

woodRock commented 3 years ago

Hi Glenn,

I am trying to create a GeoServer record for a zipfile.

I have copied the zip file in the /data/niwa/publish/ directory. It is now available via this URL (http://www.nzodn.nz/data/bio/bio.zip). I have also created a schema (bio) with a table (biological_map) on the Postgres database. This stores the filename and URL for this zip. The database does not need to store any other data. I have then created a store and layer on GeoServer for this schema.

Is this all that is required to get have working downloads on the GeoServer end?

Cheers, --

woodRock commented 3 years ago

In order to be visible on NZODN, we need to create a GeoNetwork record. A separate issue has been created for this #12.