UGS-GIO / rockcore

UGS Core Research Center Inventory web application that compiles a large data set of rock core data and displays it on an interactive web map
https://geology.utah.gov/apps/rockcore/
0 stars 0 forks source link

Photograph Database and Handling #20

Open PNatCoreCenter opened 4 years ago

PNatCoreCenter commented 4 years ago

I believe that we have a photograph storage, handling and maintenance problem

  1. Core Photograph Storage. We currently have a very large core photograph locations that we are and will be working with on a weekly basis. I know of at least two storage locations for core photographs.
    1: Google Drive (G drive); all core photographs were moved to the G drive from the U drive to increase the volume storage limits on the DTS hosted servers when Google Drive stream was implemented at the UGS. We keep all copies of the photographs here where they are backed up on a regular basis. The directories are still large and take time to download. We keep copies of the oriented JPG files and the original RAW files on the G drive.
    2: Web Server (W: drive); all web hosted .JPG core photographs are uploaded to this directory on the web server. The .RAW files are not uploaded to the web server.
  2. I'm sure that there are other core photograph storage locations at the UGS.
  3. Errors can be introduced into one or both of the images and metadata because of excessive handling.

Problem: It is bad database policy to maintain multiple databases of the same data. It can introduce errors into one or both of the storage locations, requires twice the labor and handling. There should be one database that users can upload files and indexes of new data and one database that all users (including the web server) access to view or download images. The database should be able to manage its own storage and backups.

Solution: A SQL-based database solution will manage storage and access. It will provide secure, multi-user access from multiple applications. Typical SQL servers (MYSQL, POSTREGSQL, SQL Server, etc) can handle 150 - 300 simultaneous connections on low-end web servers to several thousand on high-end web servers. For instance; a user connects to the UCRC map server and views an area of core or cuttings. As the location and inventory data is being display, multiple connections can be established to the image database and the thumb-nails, high-resolution photographs, and core documents can be cached on the web server and be available to display or download. This can be performed for one or multiple samples displayed on the screen. At the same time, new photographs and data can be uploaded and verified to the database. One location, one set of tools to access and one place to add new data. There is one document and image database to manage rather than several.

Create Thumbnails of existing and new core photographs. Problem: We tried this a few years back with a few core directories and it is VERY labor intensive to create and manage thumbnail images. At least right now, it has to be done in Photoshop one file at a time. Each photograph has to be reviewed and then processed. I believe that Photoshop has a way to Batch process images but I don't know if that will run several processes on one image, (for example, lighten, detail, rotate) or if you can process several images for one or more files.

Solution: There are several document and image storage solutions out there that allow you to process the images as they are uploaded to the database. You can create thumbnails of different sizes, crop, rotate, etc. The images are then stored as the original and processed files. You can then use either file.

Vandenberg5 commented 4 years ago

Peter's responsibilities:

1) Oversee the taking of core photos 2) Make sure photos meet standards and are consistent 3) Archive RAW images (and possible jpg images) somewhere on a backed-up drive 4) Provide usable, correctly formatted, jpg files to GIO for use in Rock Core app (make sure photos are oriented, an appropriate file size, name correctly, etc.)

GIO responsibilities: 1) receive processed jpg photos from Peter (presumably on a shared drive somewhere) 2) make sure photos display correctly on the Rock Core app 3) if there are display problems, work with Peter to find solution