ingest 2014 big_bend wv2 files

7yl4r commented 6 years ago

Following up from https://github.com/USF-IMARS/wv2-processing/issues/3 :

file list from example granule:

WV02_20141213164006_103001003B277100_14DEC13164006-M1BS-500534938020_01_P005.rename
WV02_20141213164006_103001003B277100_14DEC13164006-M1BS-500534938020_01_P005.tar
WV02_20141213164006_103001003B277100_14DEC13164006-M1BS-500534938020_01_P005.xml
WV02_20141213164007_103001003B277100_14DEC13164007-M1BS-500534938020_01_P006-BROWSE.jpg
WV02_20141213164007_103001003B277100_14DEC13164007-M1BS-500534938020_01_P006.ntf

We are only going to ingest the ntf & xml files

7yl4r commented 6 years ago

ntf files first:

product_id=11 is m1bs ntf. status_id=2 is temporary staging. area_id=6 is big_bend.

python3 -m imars_etl 
    -v 
    load 
    -d /srv/imars-objects/big_bend/wv2/2014 
    -p 11 
    -j '{"status_id":2, "area_id":6}' 
    --storage_driver no_upload 
    -l "WV02_%Y%m%d%H%M%S_{junk}-M1BS-{idNumber:12}_{otherNum:2}_P{passNumber:0>3d}.ntf"

ingests in progress look good:

MariaDB [imars_product_metadata]> SELECT filepath,date_time,product_id,area_id,status_id FROM file WHERE area_id=6;
+-----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------+-----------+
| filepath                                                                                                              | date_time           | product_id | area_id | status_id |
+-----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------+-----------+
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162814_103001002DB98000_14FEB10162814-M1BS-500534935020_01_P001.ntf | 2014-02-10 16:28:14 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162815_103001002DB98000_14FEB10162815-M1BS-500534935020_01_P002.ntf | 2014-02-10 16:28:15 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162816_103001002DB98000_14FEB10162816-M1BS-500534935020_01_P003.ntf | 2014-02-10 16:28:16 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162817_103001002DB98000_14FEB10162817-M1BS-500534935020_01_P004.ntf | 2014-02-10 16:28:17 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162818_103001002DB98000_14FEB10162818-M1BS-500534935020_01_P005.ntf | 2014-02-10 16:28:18 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162819_103001002DB98000_14FEB10162819-M1BS-500534935020_01_P006.ntf | 2014-02-10 16:28:19 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162820_103001002DB98000_14FEB10162820-M1BS-500534935020_01_P007.ntf | 2014-02-10 16:28:20 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140218163414_103001002E0EB600_14FEB18163414-M1BS-500534956040_01_P001.ntf | 2014-02-18 16:34:14 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140218163415_103001002E0EB600_14FEB18163415-M1BS-500534956040_01_P002.ntf | 2014-02-18 16:34:15 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140218163416_103001002E0EB600_14FEB18163416-M1BS-500534956040_01_P003.ntf | 2014-02-18 16:34:16 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140218163417_103001002E0EB600_14FEB18163417-M1BS-500534956040_01_P004.ntf | 2014-02-18 16:34:17 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170547_103001002C05D600_14FEB28170547-M1BS-500109379010_01_P001.ntf | 2014-02-28 17:05:47 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170548_103001002C05D600_14FEB28170548-M1BS-500109379010_01_P002.ntf | 2014-02-28 17:05:48 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170549_103001002C05D600_14FEB28170549-M1BS-500109379010_01_P003.ntf | 2014-02-28 17:05:49 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170551_103001002C05D600_14FEB28170551-M1BS-500109379010_01_P004.ntf | 2014-02-28 17:05:51 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170552_103001002C05D600_14FEB28170552-M1BS-500109379010_01_P005.ntf | 2014-02-28 17:05:52 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170553_103001002C05D600_14FEB28170553-M1BS-500109379010_01_P006.ntf | 2014-02-28 17:05:53 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170554_103001002C05D600_14FEB28170554-M1BS-500109379010_01_P007.ntf | 2014-02-28 17:05:54 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170555_103001002C05D600_14FEB28170555-M1BS-500109379010_01_P008.ntf | 2014-02-28 17:05:55 |         11 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140228170556_103001002C05D600_14FEB28170556-M1BS-500109379010_01_P009.ntf | 2014-02-28 17:05:56 |         11 |       6 |         2 |
+-----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------+-----------+

...

INFO:imars_etl.Load.Load._load_dir:201 files loaded, 800 skipped, 0 duplicates.

7yl4r commented 6 years ago

now the xml (product_id=14)

python3 -m imars_etl 
    -v 
    load 
    -d /srv/imars-objects/big_bend/wv2/2014 
    -p 14
    -j '{"status_id":2, "area_id":6}' 
    --storage_driver no_upload 
    -l "WV02_%Y%m%d%H%M%S_{junk}-M1BS-{idNumber:12}_{otherNum:2}_P{passNumber:0>3d}.xml"

...

MariaDB [imars_product_metadata]> SELECT filepath,date_time,product_id,area_id,status_id FROM file WHERE area_id=6 AND product_id=14;+-----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------+-----------+
| filepath                                                                                                              | date_time           | product_id | area_id | status_id |
+-----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------+-----------+
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162814_103001002DB98000_14FEB10162814-M1BS-500534935020_01_P001.xml | 2014-02-10 16:28:14 |         14 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162815_103001002DB98000_14FEB10162815-M1BS-500534935020_01_P002.xml | 2014-02-10 16:28:15 |         14 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162816_103001002DB98000_14FEB10162816-M1BS-500534935020_01_P003.xml | 2014-02-10 16:28:16 |         14 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162817_103001002DB98000_14FEB10162817-M1BS-500534935020_01_P004.xml | 2014-02-10 16:28:17 |         14 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162818_103001002DB98000_14FEB10162818-M1BS-500534935020_01_P005.xml | 2014-02-10 16:28:18 |         14 |       6 |         2 |
| /srv/imars-objects/big_bend/wv2/2014/WV02_20140210162819_103001002DB98000_14FEB10162819-M1BS-500534935020_01_P006.xml | 2014-02-10 16:28:19 |         14 |       6 |         2 |
+-----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------+-----------+

...

INFO:imars_etl.Load.Load._load_dir:200 files loaded, 801 skipped, 0 duplicates.

7yl4r commented 6 years ago

201 vs 200: uh oh... looks like something is mismatched. Let's try to find it:

MariaDB [imars_product_metadata]> SELECT COUNT(*),date_time FROM file WHERE status_id=2 GROUP BY date_time ORDER BY COUNT(*);
+----------+---------------------+
| COUNT(*) | date_time           |
+----------+---------------------+
|        1 | 2014-12-13 16:40:07 |
|        2 | 2014-02-10 16:28:17 |
|        2 | 2014-02-28 17:05:47 |
|        2 | 2014-02-28 17:05:56 |
...
|        2 | 2014-12-13 16:39:50 |
|        2 | 2014-12-13 16:40:05 |
|        4 | 2014-12-13 16:39:54 |
|        4 | 2014-03-01 16:29:30 |
|        4 | 2014-07-17 16:41:46 |
|        4 | 2014-04-02 16:51:15 |
+----------+---------------------+
197 rows in set (0.00 sec)

Uh.. okay so 2014-12-13 16:40:07 is the ntf without an xml, but while we are here why do some dates have count=4?

MariaDB [imars_product_metadata]> SELECT * FROM file WHERE date_time="2014-04-02 16:51:15";
+-------+-----------------------------------------------------------------------------------------------------------------------------+---------------------+-------------+------------+---------+-----------+------+------------------------------------------------+
| id    | filepath                                                                                                                    | date_time           | is_day_pass | product_id | area_id | status_id | uuid | multihash                                      |
+-------+-----------------------------------------------------------------------------------------------------------------------------+---------------------+-------------+------------+---------+-----------+------+------------------------------------------------+
| 15800 | /srv/imars-objects/big_bend/wv2/2014/WV02_20140402165115_103001002FD19200_14APR02165115-M1BS-500534937010_01_P002.ntf       | 2014-04-02 16:51:15 |        NULL |         11 |       6 |         2 | NULL | QmaMkZZQfeRv4roMdjArtTwYrRxHuiDJHphTCP7NJKwJuV |
| 15801 | /srv/imars-objects/big_bend/wv2/2014/WV02_20140402165115_103001002FD19200_14APR02165115-M1BS-500534937010_01_P003.ntf       | 2014-04-02 16:51:15 |        NULL |         11 |       6 |         2 | NULL | QmUosXj2XrovctiiyH4ScY83qArxkiDSia82JQ7irLen7A |
| 16001 | /srv/imars-objects/big_bend/wv2/2014/WV02_20140402165115_103001002FD19200_14APR02165115-M1BS-500534937010_01_P002.xml       | 2014-04-02 16:51:15 |        NULL |         14 |       6 |         2 | NULL | QmWviy2F1ZpGXwj6cxdLxNUeaDjEG36FPPfiecwZSkjVUG |
| 16002 | /srv/imars-objects/big_bend/wv2/2014/WV02_20140402165115_103001002FD19200_14APR02165115-M1BS-500534937010_01_P003.xml       | 2014-04-02 16:51:15 |        NULL |         14 |       6 |         2 | NULL | QmV6AhwAdTQ12RtrsTRDB2BKg1HX8nH8SfPhubwmxxzfb1 |
+-------+-----------------------------------------------------------------------------------------------------------------------------+---------------------+-------------+------------+---------+-----------+------+------------------------------------------------+
4 rows in set (0.00 sec)

Oh jeez. Multiple granules at the exact same time? That is almost certainly going to break things.

7yl4r commented 6 years ago

clearing everything and re-ingesting now that we have USF-IMARS/imars-etl#15 out of the way.

removing what we ingested before:

MariaDB [imars_product_metadata]> SELECT COUNT(*),product_id,status_id FROM file WHERE filepath LIKE "/srv/imars_objects/big_bend/wv2/2014/%" GROUP BY product_id, status_id;
+----------+------------+-----------+
| COUNT(*) | product_id | status_id |
+----------+------------+-----------+
|      193 |         11 |         1 |
|        8 |         11 |         4 |
|        8 |         14 |         2 |
|      192 |         14 |         3 |
+----------+------------+-----------+
4 rows in set (0.02 sec)

MariaDB [imars_product_metadata]> DELETE FROM file WHERE filepath LIKE "/srv/imars_objects/big_bend/wv2/2014/%";
Query OK, 401 rows affected (0.08 sec)

Output products (metadata below) are going into backup tif_r_rs_wv2_backup_2018-08-03.

MariaDB [imars_product_metadata]> SELECT * FROM file WHERE filepath LIKE "/srv/imars_objects/big_bend/tif_r_rs_wv2/%";
+-------+----------------------------------------------------------------------+---------------------+-------------+------------+---------+-----------+------+------------------------------------------------+
| id    | filepath                                                             | date_time           | is_day_pass | product_id | area_id | status_id | uuid | multihash                                      |
+-------+----------------------------------------------------------------------+---------------------+-------------+------------+---------+-----------+------+------------------------------------------------+
| 16236 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140218T163417.tif | 2014-02-18 16:34:17 |        NULL |         37 |       6 |         3 | NULL | QmNjXyo2DbQP5vRs9uQb7eWt68bEKBTfareXrAERnJXKpD |
| 16237 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162912.tif | 2014-03-01 16:29:12 |        NULL |         37 |       6 |         3 | NULL | QmT3zq97M7jdCEoFkEUHm9Riajoe7ZzBTLqhTyBDZzvD8J |
| 16238 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162947.tif | 2014-03-01 16:29:47 |        NULL |         37 |       6 |         3 | NULL | QmRLZDf8JD8UaFWabYNzKnj11Ska8ep3wHMZShoYChF4Kw |
| 16239 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170549.tif | 2014-02-28 17:05:49 |        NULL |         37 |       6 |         3 | NULL | QmU6y7ubQMbE6c6PrazzeHUezifXxu6Av9Z6MU9cUBhNCg |
| 16240 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140314T165116.tif | 2014-03-14 16:51:16 |        NULL |         37 |       6 |         3 | NULL | QmaWZKZw5mPCyzqzRbWN4w2LNrFatKKqjSDt2ZErL42a2R |
| 16241 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162939.tif | 2014-03-01 16:29:39 |        NULL |         37 |       6 |         3 | NULL | QmQ6GKcAGgbU9CwrhpqiYAcNCCkx5nDwcsoonoEQEekETd |
| 16242 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170555.tif | 2014-02-28 17:05:55 |        NULL |         37 |       6 |         3 | NULL | QmZyAEmV1ghDESCmFY8F5PXM31HHefGH1QgHrXRkekVhfA |
| 16243 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140527T162355.tif | 2014-05-27 16:23:55 |        NULL |         37 |       6 |         3 | NULL | QmauEPepwRh34CstJP5ibHpvwAD2zCa9B5yaRCPRgMVSxw |
| 16245 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162926.tif | 2014-03-01 16:29:26 |        NULL |         37 |       6 |         3 | NULL | QmToRYDPEhidBoKGwVHyEYvFJG2R5kwgLmQPxwzXb9LV7L |
| 16246 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164117.tif | 2014-07-17 16:41:17 |        NULL |         37 |       6 |         3 | NULL | QmQ9V4VTYdFMTa2LKsRQuehtZm8DSXqNeSqnDKfYdVTRJu |
| 16247 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164119.tif | 2014-07-17 16:41:19 |        NULL |         37 |       6 |         3 | NULL | QmVCJNJBj71xKrtDpZ5uecTKdHueEiZQ6ZveT8X9qYWxc9 |
| 16248 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140701T163155.tif | 2014-07-01 16:31:55 |        NULL |         37 |       6 |         3 | NULL | QmcGog3r6yvz2CJCmjEWAekPhpLA4gBdZieiH3v6iMYQpR |
| 16249 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164123.tif | 2014-07-17 16:41:23 |        NULL |         37 |       6 |         3 | NULL | QmQ5qZmgPsvus9a5BiCLWqgzQ5besiLwNrFomjw3MtG66Q |
| 16251 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164138.tif | 2014-07-17 16:41:38 |        NULL |         37 |       6 |         3 | NULL | QmY4cnELmGW8h6aaeF3VBKhd3JsKWSBVei7sJ4TbL7Dj6U |
| 16252 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164153.tif | 2014-07-17 16:41:53 |        NULL |         37 |       6 |         3 | NULL | QmSs2HQCtM97QaTsxwRymjz3idH73jtbJ1ikyUhRZFD9Qa |
| 16253 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164126.tif | 2014-07-17 16:41:26 |        NULL |         37 |       6 |         3 | NULL | QmcR8zU2K9vcACEQ1rvkExAtWSvHc4sSs9F8CFX1sYqvcQ |
| 16254 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164202.tif | 2014-07-17 16:42:02 |        NULL |         37 |       6 |         3 | NULL | QmV1h7WSPnNkBqv5s7UE6gUnad6u4R8fu8eJp6xg8gMMJP |
| 16255 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141017T164722.tif | 2014-10-17 16:47:22 |        NULL |         37 |       6 |         3 | NULL | QmY6FLLQMS1tuYBhNgzWRtSMzwt1LLiftzWKHp1Xxni3Yc |
| 16256 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141004T162609.tif | 2014-10-04 16:26:09 |        NULL |         37 |       6 |         3 | NULL | QmYninikx2DBdPWMbhWjVJKRpaVTs1A66TFdd7jJ6Beguu |
| 16257 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164144.tif | 2014-07-17 16:41:44 |        NULL |         37 |       6 |         3 | NULL | QmUv6Vjq42HbWnu4ZdHGCRpnfcw1ygyJsrnJVnpnXYhWD4 |
| 16258 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141004T162616.tif | 2014-10-04 16:26:16 |        NULL |         37 |       6 |         3 | NULL | QmTJ4ox5EHkroQcbPX5UhMKpCxo9ZGUAzwKJNhkWCxE9ed |
| 16259 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141017T164720.tif | 2014-10-17 16:47:20 |        NULL |         37 |       6 |         3 | NULL | QmTSNeHHKWX5ispX6V6C491VhMJ2eXepyE49eY8HjefffE |
| 16260 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163946.tif | 2014-12-13 16:39:46 |        NULL |         37 |       6 |         3 | NULL | QmRFEhbC37rgk3Wexgo6bwQBYUaKGssgstT8tX1MUhStcp |
| 16261 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164159.tif | 2014-07-17 16:41:59 |        NULL |         37 |       6 |         3 | NULL | QmYDEXjzM57u1BwQEaECgUYHocqNScYBKuDwLkSwtKxSc5 |
| 16262 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163952.tif | 2014-12-13 16:39:52 |        NULL |         37 |       6 |         3 | NULL | QmVy35pJwqLwTgVDFoAbMb8dsUqCjPRqJNHg41nkjzxKqD |
| 16263 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141004T162619.tif | 2014-10-04 16:26:19 |        NULL |         37 |       6 |         3 | NULL | QmTrzSdpHwdWtRUwKuKBLRrUDDQVKujPcv6VnewteXzDSx |
| 16264 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141004T162613.tif | 2014-10-04 16:26:13 |        NULL |         37 |       6 |         3 | NULL | QmbL9pFg1vcC8NjLda9ZL19QcAnuaVsUymRftL37Zqwwnv |
| 16265 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163950.tif | 2014-12-13 16:39:50 |        NULL |         37 |       6 |         3 | NULL | QmQiwNxRvv9C51ePwTtxXDCJt6s7oS5GBpH5FjCik2741j |
| 16278 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163951.tif | 2014-12-13 16:39:51 |        NULL |         37 |       6 |         3 | NULL | QmfG1Cx3JvXKUDKpTLD6wRaKSyhaTj8W5n6u6L2MMqqRBY |
| 16279 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T164006.tif | 2014-12-13 16:40:06 |        NULL |         37 |       6 |         3 | NULL | QmRSi5gJZJh1JTDG9aAtHX561rJFyLQtG5Bn3MxWsJxr9F |
| 16280 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163947.tif | 2014-12-13 16:39:47 |        NULL |         37 |       6 |         3 | NULL | Qmd7JxBA5zRASENaBdcu3gKF7Ay3N1za2KcwfBbxLK7n1s |
| 16281 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T164003.tif | 2014-12-13 16:40:03 |        NULL |         37 |       6 |         3 | NULL | QmcMQdscGp9wh3UnNGcEs6i9Tmi841NDASixX3qko3bXf9 |
| 16282 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163948.tif | 2014-12-13 16:39:48 |        NULL |         37 |       6 |         3 | NULL | QmTf9GC5PbzmSkZTXBvzFJWcAYqkTwzpvPbAVs7YFQtDer |
| 16283 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T163953.tif | 2014-12-13 16:39:53 |        NULL |         37 |       6 |         3 | NULL | QmNLwwSYGgbmQKTqDEW95pAUp1mV3Ah1dTnSjKdwMAowYC |
| 16284 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T164005.tif | 2014-12-13 16:40:05 |        NULL |         37 |       6 |         3 | NULL | QmaURg3Ey776ebKSzS1MtDnZcxVHGfRY7m2a1wZZnwQgfQ |
| 16285 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141213T164002.tif | 2014-12-13 16:40:02 |        NULL |         37 |       6 |         3 | NULL | QmXcLG7fLv4SjkrcdWWEg2eJuDUBmTHembD36voa4jrcuS |
| 18012 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164141.tif | 2014-07-17 16:41:41 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 18029 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170548.tif | 2014-02-28 17:05:48 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 18032 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164145.tif | 2014-07-17 16:41:45 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 18038 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140320T162917.tif | 2014-03-20 16:29:17 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19568 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162902.tif | 2014-03-01 16:29:02 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19570 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170552.tif | 2014-02-28 17:05:52 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19571 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140210T162817.tif | 2014-02-10 16:28:17 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19573 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170551.tif | 2014-02-28 17:05:51 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19574 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162903.tif | 2014-03-01 16:29:03 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19575 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140320T162918.tif | 2014-03-20 16:29:18 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19578 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140210T162815.tif | 2014-02-10 16:28:15 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19584 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140210T162816.tif | 2014-02-10 16:28:16 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 19585 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162906.tif | 2014-03-01 16:29:06 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27174 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170547.tif | 2014-02-28 17:05:47 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27178 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140210T162819.tif | 2014-02-10 16:28:19 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27195 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140228T170553.tif | 2014-02-28 17:05:53 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27220 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162925.tif | 2014-03-01 16:29:25 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27227 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140527T162401.tif | 2014-05-27 16:24:01 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27233 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162946.tif | 2014-03-01 16:29:46 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27245 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162905.tif | 2014-03-01 16:29:05 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27268 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140301T162911.tif | 2014-03-01 16:29:11 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27312 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140527T162359.tif | 2014-05-27 16:23:59 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27316 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140320T162919.tif | 2014-03-20 16:29:19 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27318 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140320T162910.tif | 2014-03-20 16:29:10 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27324 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140320T162912.tif | 2014-03-20 16:29:12 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27355 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140314T165120.tif | 2014-03-14 16:51:20 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27356 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140314T165115.tif | 2014-03-14 16:51:15 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27360 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141004T162617.tif | 2014-10-04 16:26:17 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27361 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164125.tif | 2014-07-17 16:41:25 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27364 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20141017T164717.tif | 2014-10-17 16:47:17 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27375 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164203.tif | 2014-07-17 16:42:03 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
| 27386 | /srv/imars-objects/big_bend/tif_r_rs_wv2/wv2_Rrs_20140717T164156.tif | 2014-07-17 16:41:56 |        NULL |         37 |       6 |         3 | NULL | NULL                                           |
+-------+----------------------------------------------------------------------+---------------------+-------------+------------+---------+-----------+------+------------------------------------------------+

... but we are clearing it out of the db:

MariaDB [imars_product_metadata]> DELETE FROM file WHERE filepath LIKE "/srv/imars_objects/big_bend/tif_r_rs_wv2/%";
Query OK, 68 rows affected (0.06 sec)

7yl4r commented 6 years ago

Here we go again...

xml files first this time and let's go right to status_id=3

python3 -m imars_etl -v load \
    -d /srv/imars-objects/big_bend/wv2/2014 \
    -p 14 \
    -j '{"status_id":3, "area_id":6}' \
    --storage_driver no_upload \
    -l "WV02_%Y%m%d%H%M%S_{junk}-M1BS-{idNumber:12}_{otherNum:2}_P{passNumber:0>3d}.xml" \
    --metadata_file_driver wv2_xml  \
    --metadata_file "{filepath}"  \
    --nohash

...

INFO:imars_etl.Load.Load._load_dir:215 files loaded, 859 skipped, 0 duplicates.

python3 -m imars_etl -v load \
    -d /srv/imars-objects/big_bend/wv2/2014 \
    -p 11 \
    -j '{"status_id":3, "area_id":6}' \
    --storage_driver no_upload \
    -l "WV02_%Y%m%d%H%M%S_{junk}-M1BS-{idNumber:12}_{otherNum:2}_P{passNumber:0>3d}.ntf" \
    --metadata_file_driver wv2_xml  \
    --metadata_file "{directory}/{basename}.xml"  \
    --nohash

...

INFO:imars_etl.Load.Load._load_dir:215 files loaded, 859 skipped, 0 duplicates.

7yl4r commented 6 years ago

Yep, looks good:

MariaDB [imars_product_metadata]> SELECT COUNT(*),date_time FROM file WHERE area_id=6 AND product_id in (11,14) GROUP BY date_time; 
+----------+----------------------------+
| COUNT(*) | date_time                  |
+----------+----------------------------+
|        2 | 2014-02-10 16:28:14.442250 |
|        2 | 2014-02-10 16:28:15.379850 |
|        2 | 2014-02-10 16:28:16.444850 |
|        2 | 2014-02-10 16:28:17.510050 |
|        2 | 2014-02-10 16:28:18.575250 |
|        2 | 2014-02-10 16:28:19.640450 |
|        2 | 2014-02-10 16:28:20.705450 |
|        2 | 2014-02-18 16:34:14.519450 |
|        2 | 2014-02-18 16:34:15.564050 |
|        2 | 2014-02-18 16:34:16.745250 |
|        2 | 2014-02-18 16:34:17.926650 |
|        2 | 2014-02-28 17:05:47.801050 |
|        2 | 2014-02-28 17:05:48.841450 |
|        2 | 2014-02-28 17:05:49.973250 |
|        2 | 2014-02-28 17:05:51.104850 |
|        2 | 2014-02-28 17:05:52.236450 |
|        2 | 2014-02-28 17:05:53.368050 |
|        2 | 2014-02-28 17:05:54.499850 |
|        2 | 2014-02-28 17:05:55.631450 |
|        2 | 2014-02-28 17:05:56.763050 |
|        2 | 2014-02-28 17:05:57.894650 |
|        2 | 2014-03-01 16:29:02.185050 |
|        2 | 2014-03-01 16:29:03.215450 |
|        2 | 2014-03-01 16:29:04.381850 |
|        2 | 2014-03-01 16:29:05.548250 |
|        2 | 2014-03-01 16:29:06.714450 |
|        2 | 2014-03-01 16:29:07.880650 |
|        2 | 2014-03-01 16:29:09.046850 |
|        2 | 2014-03-01 16:29:10.213050 |
|        2 | 2014-03-01 16:29:11.379450 |
|        2 | 2014-03-01 16:29:12.545650 |
|        2 | 2014-03-01 16:29:21.179950 |
|        2 | 2014-03-01 16:29:22.335350 |
|        2 | 2014-03-01 16:29:23.490750 |
|        2 | 2014-03-01 16:29:24.646150 |
|        2 | 2014-03-01 16:29:25.801350 |
|        2 | 2014-03-01 16:29:26.956550 |
|        2 | 2014-03-01 16:29:28.111950 |
|        2 | 2014-03-01 16:29:29.267150 |
|        2 | 2014-03-01 16:29:30.422550 |
|        2 | 2014-03-01 16:29:30.842950 |
|        2 | 2014-03-01 16:29:37.381050 |
|        2 | 2014-03-01 16:29:38.349250 |
|        2 | 2014-03-01 16:29:39.448050 |
|        2 | 2014-03-01 16:29:40.546650 |
|        2 | 2014-03-01 16:29:41.645450 |
|        2 | 2014-03-01 16:29:42.744050 |
|        2 | 2014-03-01 16:29:43.842850 |
|        2 | 2014-03-01 16:29:44.941450 |
|        2 | 2014-03-01 16:29:46.040250 |
|        2 | 2014-03-01 16:29:47.138850 |
|        2 | 2014-03-14 16:51:14.996250 |
|        2 | 2014-03-14 16:51:15.904650 |
|        2 | 2014-03-14 16:51:16.915650 |
|        2 | 2014-03-14 16:51:17.926650 |
|        2 | 2014-03-14 16:51:18.937650 |
|        2 | 2014-03-14 16:51:19.948450 |
|        2 | 2014-03-14 16:51:20.959450 |
|        2 | 2014-03-14 16:51:21.970450 |
|        2 | 2014-03-14 16:51:22.981450 |
|        2 | 2014-03-14 16:51:23.992250 |
|        2 | 2014-03-20 16:29:09.215450 |
|        2 | 2014-03-20 16:29:10.239450 |
|        2 | 2014-03-20 16:29:11.374850 |
|        2 | 2014-03-20 16:29:12.510450 |
|        2 | 2014-03-20 16:29:13.645850 |
|        2 | 2014-03-20 16:29:14.781050 |
|        2 | 2014-03-20 16:29:15.916450 |
|        2 | 2014-03-20 16:29:17.052250 |
|        2 | 2014-03-20 16:29:18.187450 |
|        2 | 2014-03-20 16:29:19.323050 |
|        2 | 2014-04-02 16:51:04.264050 |
|        2 | 2014-04-02 16:51:05.253850 |
|        2 | 2014-04-02 16:51:06.353250 |
|        2 | 2014-04-02 16:51:14.588150 |
|        2 | 2014-04-02 16:51:15.615350 |
|        2 | 2014-04-02 16:51:15.729350 |
|        2 | 2014-04-11 16:18:51.518250 |
|        2 | 2014-04-11 16:18:52.539850 |
|        2 | 2014-04-11 16:18:53.672650 |
|        2 | 2014-04-11 16:18:54.805650 |
|        2 | 2014-04-11 16:18:55.938450 |
|        2 | 2014-04-11 16:18:57.071450 |
|        2 | 2014-04-11 16:18:58.204250 |
|        2 | 2014-04-11 16:18:59.337250 |
|        2 | 2014-04-11 16:19:00.470050 |
|        2 | 2014-04-11 16:19:01.603050 |
|        2 | 2014-04-11 16:19:10.392350 |
|        2 | 2014-04-11 16:19:11.572350 |
|        2 | 2014-04-11 16:19:12.752350 |
|        2 | 2014-04-11 16:19:13.932350 |
|        2 | 2014-04-11 16:19:15.112350 |
|        2 | 2014-04-11 16:19:16.292350 |
|        2 | 2014-04-11 16:19:17.472350 |
|        2 | 2014-04-11 16:19:18.652350 |
|        2 | 2014-04-11 16:19:19.832350 |
|        2 | 2014-04-11 16:19:20.312750 |
|        2 | 2014-05-27 16:23:55.377450 |
|        2 | 2014-05-27 16:23:56.298250 |
|        2 | 2014-05-27 16:23:57.321250 |
|        2 | 2014-05-27 16:23:58.344450 |
|        2 | 2014-05-27 16:23:59.367850 |
|        2 | 2014-05-27 16:24:00.391050 |
|        2 | 2014-05-27 16:24:01.414250 |
|        2 | 2014-06-09 16:43:49.933950 |
|        2 | 2014-06-09 16:43:51.022950 |
|        2 | 2014-06-09 16:43:52.112550 |
|        2 | 2014-06-09 16:43:53.201550 |
|        2 | 2014-06-09 16:43:54.291150 |
|        2 | 2014-06-09 16:43:55.380150 |
|        2 | 2014-06-09 16:43:56.320550 |
|        2 | 2014-06-25 16:53:09.624450 |
|        2 | 2014-06-25 16:53:10.663650 |
|        2 | 2014-06-25 16:53:11.817250 |
|        2 | 2014-06-25 16:53:12.970850 |
|        2 | 2014-06-25 16:53:14.124450 |
|        2 | 2014-06-25 16:53:15.278050 |
|        2 | 2014-06-25 16:53:16.431650 |
|        2 | 2014-06-28 16:43:00.121050 |
|        2 | 2014-06-28 16:43:01.097250 |
|        2 | 2014-06-28 16:43:02.182050 |
|        2 | 2014-06-28 16:43:03.266850 |
|        2 | 2014-06-28 16:43:04.351650 |
|        2 | 2014-06-28 16:43:05.436450 |
|        2 | 2014-06-28 16:43:06.521250 |
|        2 | 2014-06-28 16:43:07.606050 |
|        2 | 2014-06-28 16:43:08.690850 |
|        2 | 2014-06-28 16:43:09.775650 |
|        2 | 2014-07-01 16:31:49.535250 |
|        2 | 2014-07-01 16:31:50.703850 |
|        2 | 2014-07-01 16:31:51.997050 |
|        2 | 2014-07-01 16:31:53.290650 |
|        2 | 2014-07-01 16:31:54.584050 |
|        2 | 2014-07-01 16:31:55.877650 |
|        2 | 2014-07-01 16:31:57.171050 |
|        2 | 2014-07-17 16:41:16.735850 |
|        2 | 2014-07-17 16:41:17.904650 |
|        2 | 2014-07-17 16:41:19.198650 |
|        2 | 2014-07-17 16:41:20.492450 |
|        2 | 2014-07-17 16:41:21.786450 |
|        2 | 2014-07-17 16:41:23.080250 |
|        2 | 2014-07-17 16:41:24.374250 |
|        2 | 2014-07-17 16:41:25.668050 |
|        2 | 2014-07-17 16:41:26.962050 |
|        2 | 2014-07-17 16:41:28.255850 |
|        2 | 2014-07-17 16:41:37.027750 |
|        2 | 2014-07-17 16:41:38.230150 |
|        2 | 2014-07-17 16:41:39.432550 |
|        2 | 2014-07-17 16:41:40.634750 |
|        2 | 2014-07-17 16:41:41.837350 |
|        2 | 2014-07-17 16:41:43.039550 |
|        2 | 2014-07-17 16:41:44.241950 |
|        2 | 2014-07-17 16:41:45.444150 |
|        2 | 2014-07-17 16:41:46.646550 |
|        2 | 2014-07-17 16:41:46.994350 |
|        2 | 2014-07-17 16:41:53.548450 |
|        2 | 2014-07-17 16:41:54.525050 |
|        2 | 2014-07-17 16:41:55.609850 |
|        2 | 2014-07-17 16:41:56.694650 |
|        2 | 2014-07-17 16:41:57.779450 |
|        2 | 2014-07-17 16:41:58.864250 |
|        2 | 2014-07-17 16:41:59.949050 |
|        2 | 2014-07-17 16:42:01.033850 |
|        2 | 2014-07-17 16:42:02.118650 |
|        2 | 2014-07-17 16:42:03.203450 |
|        2 | 2014-10-04 16:26:09.789650 |
|        2 | 2014-10-04 16:26:10.848050 |
|        2 | 2014-10-04 16:26:12.019250 |
|        2 | 2014-10-04 16:26:13.190450 |
|        2 | 2014-10-04 16:26:14.362050 |
|        2 | 2014-10-04 16:26:15.533450 |
|        2 | 2014-10-04 16:26:16.704850 |
|        2 | 2014-10-04 16:26:17.876450 |
|        2 | 2014-10-04 16:26:19.048050 |
|        2 | 2014-10-04 16:26:20.219450 |
|        2 | 2014-10-17 16:47:12.503650 |
|        2 | 2014-10-17 16:47:13.490050 |
|        2 | 2014-10-17 16:47:14.585650 |
|        2 | 2014-10-17 16:47:15.681250 |
|        2 | 2014-10-17 16:47:16.776650 |
|        2 | 2014-10-17 16:47:17.872250 |
|        2 | 2014-10-17 16:47:18.967650 |
|        2 | 2014-10-17 16:47:20.063450 |
|        2 | 2014-10-17 16:47:21.158850 |
|        2 | 2014-10-17 16:47:22.254450 |
|        2 | 2014-12-13 16:39:45.274950 |
|        2 | 2014-12-13 16:39:46.467550 |
|        2 | 2014-12-13 16:39:47.660350 |
|        2 | 2014-12-13 16:39:48.853150 |
|        2 | 2014-12-13 16:39:50.046150 |
|        2 | 2014-12-13 16:39:51.238950 |
|        2 | 2014-12-13 16:39:52.431550 |
|        2 | 2014-12-13 16:39:53.624350 |
|        2 | 2014-12-13 16:39:54.817150 |
|        2 | 2014-12-13 16:39:54.934150 |
|        2 | 2014-12-13 16:40:01.658250 |
|        2 | 2014-12-13 16:40:02.760450 |
|        2 | 2014-12-13 16:40:03.982450 |
|        2 | 2014-12-13 16:40:05.204250 |
|        2 | 2014-12-13 16:40:06.426050 |
|        2 | 2014-12-13 16:40:07.647850 |
|        2 | 2014-12-13 16:40:08.869850 |
|        2 | 2014-12-13 16:40:10.091650 |
|        2 | 2014-12-13 16:40:11.313450 |
|        2 | 2014-12-13 16:40:12.535250 |
|        2 | 2014-12-27 16:23:39.690350 |
|        2 | 2014-12-27 16:23:40.871750 |
|        2 | 2014-12-27 16:23:42.053150 |
|        2 | 2014-12-27 16:23:43.234550 |
|        2 | 2014-12-27 16:23:43.285750 |
|        2 | 2014-12-27 16:23:55.609450 |
|        2 | 2014-12-27 16:23:56.696650 |
|        2 | 2014-12-27 16:23:57.783850 |
|        2 | 2014-12-27 16:23:58.871250 |
|        2 | 2014-12-27 16:23:59.958450 |
+----------+----------------------------+
215 rows in set (0.00 sec)

MariaDB [imars_product_metadata]> SELECT COUNT(*),date_time FROM file WHERE area_id=6 AND product_id in (11,14) GROUP BY date_time ORDER BY date_time;
+----------+----------------------------+
| COUNT(*) | date_time                  |
+----------+----------------------------+
|        2 | 2014-02-10 16:28:14.442250 |
|        2 | 2014-02-10 16:28:15.379850 |
|        2 | 2014-02-10 16:28:16.444850 |
|        2 | 2014-02-10 16:28:17.510050 |
|        2 | 2014-02-10 16:28:18.575250 |
|        2 | 2014-02-10 16:28:19.640450 |
|        2 | 2014-02-10 16:28:20.705450 |
|        2 | 2014-02-18 16:34:14.519450 |
|        2 | 2014-02-18 16:34:15.564050 |
|        2 | 2014-02-18 16:34:16.745250 |
|        2 | 2014-02-18 16:34:17.926650 |
|        2 | 2014-02-28 17:05:47.801050 |
|        2 | 2014-02-28 17:05:48.841450 |
|        2 | 2014-02-28 17:05:49.973250 |
|        2 | 2014-02-28 17:05:51.104850 |
|        2 | 2014-02-28 17:05:52.236450 |
|        2 | 2014-02-28 17:05:53.368050 |
|        2 | 2014-02-28 17:05:54.499850 |
|        2 | 2014-02-28 17:05:55.631450 |
|        2 | 2014-02-28 17:05:56.763050 |
|        2 | 2014-02-28 17:05:57.894650 |
|        2 | 2014-03-01 16:29:02.185050 |
|        2 | 2014-03-01 16:29:03.215450 |
|        2 | 2014-03-01 16:29:04.381850 |
|        2 | 2014-03-01 16:29:05.548250 |
|        2 | 2014-03-01 16:29:06.714450 |
|        2 | 2014-03-01 16:29:07.880650 |
|        2 | 2014-03-01 16:29:09.046850 |
|        2 | 2014-03-01 16:29:10.213050 |
|        2 | 2014-03-01 16:29:11.379450 |
|        2 | 2014-03-01 16:29:12.545650 |
|        2 | 2014-03-01 16:29:21.179950 |
|        2 | 2014-03-01 16:29:22.335350 |
|        2 | 2014-03-01 16:29:23.490750 |
|        2 | 2014-03-01 16:29:24.646150 |
|        2 | 2014-03-01 16:29:25.801350 |
|        2 | 2014-03-01 16:29:26.956550 |
|        2 | 2014-03-01 16:29:28.111950 |
|        2 | 2014-03-01 16:29:29.267150 |
|        2 | 2014-03-01 16:29:30.422550 |
|        2 | 2014-03-01 16:29:30.842950 |
|        2 | 2014-03-01 16:29:37.381050 |
|        2 | 2014-03-01 16:29:38.349250 |
|        2 | 2014-03-01 16:29:39.448050 |
|        2 | 2014-03-01 16:29:40.546650 |
|        2 | 2014-03-01 16:29:41.645450 |
|        2 | 2014-03-01 16:29:42.744050 |
|        2 | 2014-03-01 16:29:43.842850 |
|        2 | 2014-03-01 16:29:44.941450 |
|        2 | 2014-03-01 16:29:46.040250 |
|        2 | 2014-03-01 16:29:47.138850 |
|        2 | 2014-03-14 16:51:14.996250 |
|        2 | 2014-03-14 16:51:15.904650 |
|        2 | 2014-03-14 16:51:16.915650 |
|        2 | 2014-03-14 16:51:17.926650 |
|        2 | 2014-03-14 16:51:18.937650 |
|        2 | 2014-03-14 16:51:19.948450 |
|        2 | 2014-03-14 16:51:20.959450 |
|        2 | 2014-03-14 16:51:21.970450 |
|        2 | 2014-03-14 16:51:22.981450 |
|        2 | 2014-03-14 16:51:23.992250 |
|        2 | 2014-03-20 16:29:09.215450 |
|        2 | 2014-03-20 16:29:10.239450 |
|        2 | 2014-03-20 16:29:11.374850 |
|        2 | 2014-03-20 16:29:12.510450 |
|        2 | 2014-03-20 16:29:13.645850 |
|        2 | 2014-03-20 16:29:14.781050 |
|        2 | 2014-03-20 16:29:15.916450 |
|        2 | 2014-03-20 16:29:17.052250 |
|        2 | 2014-03-20 16:29:18.187450 |
|        2 | 2014-03-20 16:29:19.323050 |
|        2 | 2014-04-02 16:51:04.264050 |
|        2 | 2014-04-02 16:51:05.253850 |
|        2 | 2014-04-02 16:51:06.353250 |
|        2 | 2014-04-02 16:51:14.588150 |
|        2 | 2014-04-02 16:51:15.615350 |
|        2 | 2014-04-02 16:51:15.729350 |
|        2 | 2014-04-11 16:18:51.518250 |
|        2 | 2014-04-11 16:18:52.539850 |
|        2 | 2014-04-11 16:18:53.672650 |
|        2 | 2014-04-11 16:18:54.805650 |
|        2 | 2014-04-11 16:18:55.938450 |
|        2 | 2014-04-11 16:18:57.071450 |
|        2 | 2014-04-11 16:18:58.204250 |
|        2 | 2014-04-11 16:18:59.337250 |
|        2 | 2014-04-11 16:19:00.470050 |
|        2 | 2014-04-11 16:19:01.603050 |
|        2 | 2014-04-11 16:19:10.392350 |
|        2 | 2014-04-11 16:19:11.572350 |
|        2 | 2014-04-11 16:19:12.752350 |
|        2 | 2014-04-11 16:19:13.932350 |
|        2 | 2014-04-11 16:19:15.112350 |
|        2 | 2014-04-11 16:19:16.292350 |
|        2 | 2014-04-11 16:19:17.472350 |
|        2 | 2014-04-11 16:19:18.652350 |
|        2 | 2014-04-11 16:19:19.832350 |
|        2 | 2014-04-11 16:19:20.312750 |
|        2 | 2014-05-27 16:23:55.377450 |
|        2 | 2014-05-27 16:23:56.298250 |
|        2 | 2014-05-27 16:23:57.321250 |
|        2 | 2014-05-27 16:23:58.344450 |
|        2 | 2014-05-27 16:23:59.367850 |
|        2 | 2014-05-27 16:24:00.391050 |
|        2 | 2014-05-27 16:24:01.414250 |
|        2 | 2014-06-09 16:43:49.933950 |
|        2 | 2014-06-09 16:43:51.022950 |
|        2 | 2014-06-09 16:43:52.112550 |
|        2 | 2014-06-09 16:43:53.201550 |
|        2 | 2014-06-09 16:43:54.291150 |
|        2 | 2014-06-09 16:43:55.380150 |
|        2 | 2014-06-09 16:43:56.320550 |
|        2 | 2014-06-25 16:53:09.624450 |
|        2 | 2014-06-25 16:53:10.663650 |
|        2 | 2014-06-25 16:53:11.817250 |
|        2 | 2014-06-25 16:53:12.970850 |
|        2 | 2014-06-25 16:53:14.124450 |
|        2 | 2014-06-25 16:53:15.278050 |
|        2 | 2014-06-25 16:53:16.431650 |
|        2 | 2014-06-28 16:43:00.121050 |
|        2 | 2014-06-28 16:43:01.097250 |
|        2 | 2014-06-28 16:43:02.182050 |
|        2 | 2014-06-28 16:43:03.266850 |
|        2 | 2014-06-28 16:43:04.351650 |
|        2 | 2014-06-28 16:43:05.436450 |
|        2 | 2014-06-28 16:43:06.521250 |
|        2 | 2014-06-28 16:43:07.606050 |
|        2 | 2014-06-28 16:43:08.690850 |
|        2 | 2014-06-28 16:43:09.775650 |
|        2 | 2014-07-01 16:31:49.535250 |
|        2 | 2014-07-01 16:31:50.703850 |
|        2 | 2014-07-01 16:31:51.997050 |
|        2 | 2014-07-01 16:31:53.290650 |
|        2 | 2014-07-01 16:31:54.584050 |
|        2 | 2014-07-01 16:31:55.877650 |
|        2 | 2014-07-01 16:31:57.171050 |
|        2 | 2014-07-17 16:41:16.735850 |
|        2 | 2014-07-17 16:41:17.904650 |
|        2 | 2014-07-17 16:41:19.198650 |
|        2 | 2014-07-17 16:41:20.492450 |
|        2 | 2014-07-17 16:41:21.786450 |
|        2 | 2014-07-17 16:41:23.080250 |
|        2 | 2014-07-17 16:41:24.374250 |
|        2 | 2014-07-17 16:41:25.668050 |
|        2 | 2014-07-17 16:41:26.962050 |
|        2 | 2014-07-17 16:41:28.255850 |
|        2 | 2014-07-17 16:41:37.027750 |
|        2 | 2014-07-17 16:41:38.230150 |
|        2 | 2014-07-17 16:41:39.432550 |
|        2 | 2014-07-17 16:41:40.634750 |
|        2 | 2014-07-17 16:41:41.837350 |
|        2 | 2014-07-17 16:41:43.039550 |
|        2 | 2014-07-17 16:41:44.241950 |
|        2 | 2014-07-17 16:41:45.444150 |
|        2 | 2014-07-17 16:41:46.646550 |
|        2 | 2014-07-17 16:41:46.994350 |
|        2 | 2014-07-17 16:41:53.548450 |
|        2 | 2014-07-17 16:41:54.525050 |
|        2 | 2014-07-17 16:41:55.609850 |
|        2 | 2014-07-17 16:41:56.694650 |
|        2 | 2014-07-17 16:41:57.779450 |
|        2 | 2014-07-17 16:41:58.864250 |
|        2 | 2014-07-17 16:41:59.949050 |
|        2 | 2014-07-17 16:42:01.033850 |
|        2 | 2014-07-17 16:42:02.118650 |
|        2 | 2014-07-17 16:42:03.203450 |
|        2 | 2014-10-04 16:26:09.789650 |
|        2 | 2014-10-04 16:26:10.848050 |
|        2 | 2014-10-04 16:26:12.019250 |
|        2 | 2014-10-04 16:26:13.190450 |
|        2 | 2014-10-04 16:26:14.362050 |
|        2 | 2014-10-04 16:26:15.533450 |
|        2 | 2014-10-04 16:26:16.704850 |
|        2 | 2014-10-04 16:26:17.876450 |
|        2 | 2014-10-04 16:26:19.048050 |
|        2 | 2014-10-04 16:26:20.219450 |
|        2 | 2014-10-17 16:47:12.503650 |
|        2 | 2014-10-17 16:47:13.490050 |
|        2 | 2014-10-17 16:47:14.585650 |
|        2 | 2014-10-17 16:47:15.681250 |
|        2 | 2014-10-17 16:47:16.776650 |
|        2 | 2014-10-17 16:47:17.872250 |
|        2 | 2014-10-17 16:47:18.967650 |
|        2 | 2014-10-17 16:47:20.063450 |
|        2 | 2014-10-17 16:47:21.158850 |
|        2 | 2014-10-17 16:47:22.254450 |
|        2 | 2014-12-13 16:39:45.274950 |
|        2 | 2014-12-13 16:39:46.467550 |
|        2 | 2014-12-13 16:39:47.660350 |
|        2 | 2014-12-13 16:39:48.853150 |
|        2 | 2014-12-13 16:39:50.046150 |
|        2 | 2014-12-13 16:39:51.238950 |
|        2 | 2014-12-13 16:39:52.431550 |
|        2 | 2014-12-13 16:39:53.624350 |
|        2 | 2014-12-13 16:39:54.817150 |
|        2 | 2014-12-13 16:39:54.934150 |
|        2 | 2014-12-13 16:40:01.658250 |
|        2 | 2014-12-13 16:40:02.760450 |
|        2 | 2014-12-13 16:40:03.982450 |
|        2 | 2014-12-13 16:40:05.204250 |
|        2 | 2014-12-13 16:40:06.426050 |
|        2 | 2014-12-13 16:40:07.647850 |
|        2 | 2014-12-13 16:40:08.869850 |
|        2 | 2014-12-13 16:40:10.091650 |
|        2 | 2014-12-13 16:40:11.313450 |
|        2 | 2014-12-13 16:40:12.535250 |
|        2 | 2014-12-27 16:23:39.690350 |
|        2 | 2014-12-27 16:23:40.871750 |
|        2 | 2014-12-27 16:23:42.053150 |
|        2 | 2014-12-27 16:23:43.234550 |
|        2 | 2014-12-27 16:23:43.285750 |
|        2 | 2014-12-27 16:23:55.609450 |
|        2 | 2014-12-27 16:23:56.696650 |
|        2 | 2014-12-27 16:23:57.783850 |
|        2 | 2014-12-27 16:23:58.871250 |
|        2 | 2014-12-27 16:23:59.958450 |
+----------+----------------------------+
215 rows in set (0.00 sec)

USF-IMARS / wv-land-cover

ingest 2014 big_bend wv2 files #8