rtrwalker / geotecha

A software suite for geotechnical engineering
GNU General Public License v3.0
27 stars 16 forks source link

AGS #4

Open pedromorgan opened 6 years ago

pedromorgan commented 6 years ago

Quick question.. as i'm a backend developer..

are u familiar/use/know the AGS file format ??

= http://www.ags.org.uk/data-format/ = and stuff I'm up to https://github.com/open-geotechnical/

there's a whole minefiled am walking into of spreadsheets, vs csv, vs et all with json online = future

rtrwalker commented 6 years ago

I've never explicitly had to deal with AGS file format, but I have a vague knowledge of what it is about. I think of it as a standardized way of naming the each table (and its columns/fields) in a relational database. Because all the naming is consistent only the data has to be shared in a data file; the relationships between the tables is defined by the AGS standard.

Any of the spreadsheets, csv etc are just ways to record/transfer data so that it can get put in an AGS compatible database. So you might have multiple text files, or worksheets in a spreadsheet, each of which represents a table in the database. The name of each csv/worksheet is the table name, 1st row is the table headers, then there is data on each row. The AGS format itself is just a way to put everything in a single text file.

A small example is at http://blogs.rand.com/civil/2015/05/geotechnical-data-formats.html . The data structure gets a bit convoluted (hence the need for a standardised format) because geo-data is itself convoluted. Say I drill a borehole and take 5 samples at different depth. On each of the samples I do two types of tests. One test produces a single number I have to store, the other produces multiple values I need to store (e.g. some quantity varying over time). There needs to be a separate database table for each of those items. I think AGS format is a bit overwhelming at first glance because they have gone, "What is every possible piece of geo-data we might come across, how does it all relate and how should we name every single piece of it?" Most people would only use a subset of the tables. Sometimes they haven't thought of everything so some organisations will build on/ customize the AGS format (e.g. the Roads and Maritime Services in NSW Australia: http://www.rms.nsw.gov.au/business-industry/partners-suppliers/design-documents/geotechnical-information.html )

If I was making something to parse the AGS data I would only consider the single text file version. It is someone else's responsibility to get their data into the AGS format in the first place.