search

capstone-coal / coal-sds

An Apache OODT-powered Science Data System for COAL
Apache License 2.0
2 stars 3 forks source link

Implement OODT Crawler for injecting staged products into File Manager #8

Closed lewismc closed 6 years ago

lewismc commented 6 years ago

Once we have implemented #7 we need to look at implementing OODT's Crawler component for injecting staged products into the File Manager.

lewismc commented 6 years ago

This can be done as follows

$ cd coal-sds-deploy/crawler/bin
$ ./bin/crawler_launcher --filemgrUrl http://localhost:9000 --operation --launchMetCrawler --clientTransferer org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory --productPath /usr/local/coal-sds-deploy/data/staging --metExtractor org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor --metExtractorConfig /usr/local/coal-sds-deploy/data/met/tika.conf

The output is as follows

Bad level value for property: org.apache.oodt.cas.crawl.typedetection.level
Bad level value for property: org.apache.oodt.cas.crawl.config.level
Bad level value for property: org.apache.oodt.cas.crawl.level
Bad level value for property: org.apache.oodt.cas.crawl.action.level
Bad level value for property: org.apache.oodt.cas.crawl.util.level
Setting property 'StdProductCrawler.filemgrUrl'
Setting property 'MetExtractorProductCrawler.filemgrUrl'
Setting property 'AutoDetectProductCrawler.filemgrUrl'
Setting property 'MetExtractorProductCrawler.metExtractor'
Setting property 'MetExtractorProductCrawler.metExtractorConfig'
Setting property 'StdProductCrawler.clientTransferer'
Setting property 'MetExtractorProductCrawler.clientTransferer'
Setting property 'AutoDetectProductCrawler.clientTransferer'
Setting property 'StdProductCrawler.productPath'
Setting property 'MetExtractorProductCrawler.productPath'
Setting property 'AutoDetectProductCrawler.productPath'
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'StdProductCrawler.productPath' set to value [/usr/local/coal-sds-deploy/data/staging]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'MetExtractorProductCrawler.metExtractorConfig' set to value [/usr/local/coal-sds-deploy/data/met/tika.conf]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'MetExtractorProductCrawler.clientTransferer' set to value [org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'AutoDetectProductCrawler.filemgrUrl' set to value [http://localhost:9000]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'AutoDetectProductCrawler.clientTransferer' set to value [org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'StdProductCrawler.filemgrUrl' set to value [http://localhost:9000]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'AutoDetectProductCrawler.productPath' set to value [/usr/local/coal-sds-deploy/data/staging]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'MetExtractorProductCrawler.metExtractor' set to value [org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'MetExtractorProductCrawler.filemgrUrl' set to value [http://localhost:9000]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'StdProductCrawler.clientTransferer' set to value [org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory]
Sep 28, 2018 2:31:31 PM org.springframework.beans.factory.config.PropertyOverrideConfigurer processKey
FINE: Property 'MetExtractorProductCrawler.productPath' set to value [/usr/local/coal-sds-deploy/data/staging]
Sep 28, 2018 2:31:31 PM org.apache.oodt.cas.crawl.ProductCrawler crawl
INFO: Crawling /usr/local/coal-sds-deploy/data/staging
Sep 28, 2018 2:31:31 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
INFO: Handling file /usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr
Sep 28, 2018 2:31:31 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Invoking tika extractor on file [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Number of captured tika metadata keys: [4]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [X-Parsed-By] with value [org.apache.tika.parser.DefaultParser]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [Content-Encoding] with value [ISO-8859-1]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [content] with value [ENVI
description = {
  Georeferenced Image built from input GLT. [Tue Aug 11 15:39:46 2015] [Tue
  Aug 11 16:17:51 2015]}
samples = 1363
lines = 22826
bands = 432
header offset = 0
file type = ENVI Standard
data type = 4
interleave = bil
sensor type = Unknown
byte order = 0
map info = { UTM , 1.000 , 1.000 , 201603.137 , 4061363.983 , 2.7000000000e+00 , 2.7000000000e+00 , 13 , North , WGS-84 , units=Meters , rotation=90.00000000 }
wavelength units = Nanometers
correction factors = { 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.880586 , 1.741631 , 1.524632 , 1.564904 , 1.390035 , 1.342501 , 1.264768 , 1.200357 , 1.131471 , 1.06512 , 1.105211 , 1.081234 , 1.064999 , 1.056798 , 1.042573 , 1.009621 , 1.009031 , 1.002107 , 0.997332 , 0.976807 , 0.963755 , 0.969028 , 0.962823 , 0.949522 , 0.960568 , 0.957813 , 0.934766 , 0.944994 , 0.937726 , 0.935257 , 0.932706 , 0.932568 , 0.933217 , 0.928705 , 0.929294 , 0.936669 , 0.935498 , 0.94823 , 0.949846 , 0.945885 , 0.935468 , 0.930084 , 0.934473 , 0.935378 , 0.939193 , 0.935081 , 0.937398 , 0.943396 , 0.947133 , 0.950645 , 0.945531 , 0.940295 , 0.933129 , 0.930664 , 0.92736 , 0.931786 , 0.928217 , 0.928205 , 0.926481 , 0.928583 , 0.930504 , 0.93648 , 0.930731 , 0.931265 , 0.935063 , 0.93434 , 0.926983 , 0.932689 , 0.936477 , 0.939647 , 0.940155 , 0.937519 , 0.939448 , 0.942124 , 0.93653 , 0.9435 , 0.959204 , 0.942566 , 0.940873 , 0.939414 , 0.939822 , 0.940174 , 0.941372 , 0.939347 , 0.942108 , 0.942664 , 0.934811 , 0.934567 , 0.937712 , 0.940611 , 0.944809 , 0.939877 , 0.943376 , 0.939189 , 0.943619 , 0.946268 , 0.940166 , 0.953752 , 0.958975 , 0.954512 , 0.954103 , 0.958978 , 0.953247 , 0.952199 , 0.956082 , 0.957846 , 0.970078 , 0.973704 , 0.980014 , 0.928845 , 0.922973 , 0.954414 , 0.95521 , 0.961276 , 0.964513 , 0.965296 , 0.964644 , 0.954999 , 0.951133 , 0.956216 , 0.951977 , 0.948547 , 0.949499 , 0.952685 , 0.950158 , 0.944263 , 0.936946 , 0.938394 , 0.941325 , 0.94116 , 0.941397 , 0.940811 , 0.942695 , 0.945228 , 0.953929 , 0.962457 , 0.968728 , 0.963947 , 0.961222 , 0.963003 , 0.967658 , 0.969773 , 0.970294 , 0.963456 , 0.970497 , 0.976972 , 0.961611 , 0.953081 , 0.945668 , 0.993867 , 1.019915 , 0.997013 , 0.977643 , 0.998022 , 1.007041 , 1.003881 , 0.991335 , 0.976202 , 0.967636 , 0.969294 , 0.965331 , 0.968705 , 0.965705 , 0.973601 , 0.97282 , 0.970848 , 0.970687 , 0.969394 , 0.972263 , 0.969286 , 0.970327 , 0.97754 , 0.984703 , 0.993916 , 1.02186 , 1.054704 , 1.061183 , 1.036962 , 1.012519 , 0.991209 , 0.975974 , 0.965446 , 0.958801 , 0.951519 , 0.960628 , 0.957276 , 0.96061 , 0.95689 , 0.956666 , 0.965567 , 0.982251 , 1.038526 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.131643 , 1.150333 , 1.112466 , 1.06255 , 1.033182 , 1.03018 , 1.006512 , 0.993285 , 1.003557 , 1.014441 , 0.998586 , 0.994426 , 0.987975 , 0.984681 , 0.981863 , 0.964106 , 0.956432 , 0.954467 , 0.956429 , 0.950668 , 0.947217 , 0.944635 , 0.942942 , 0.941078 , 0.943502 , 0.950408 , 0.957625 , 0.965073 , 0.976012 , 0.976683 , 0.975625 , 0.968011 , 0.968843 , 0.970632 , 0.960977 , 0.960505 , 0.955015 , 0.953597 , 0.951119 , 0.945679 , 0.949988 , 0.951236 , 0.947813 , 0.948004 , 0.950015 , 0.939258 , 0.945863 , 0.953927 , 0.953145 , 0.945291 , 0.942319 , 0.947022 , 0.948264 , 0.947112 , 0.942092 , 0.943128 , 0.948068 , 0.944432 , 0.950396 , 0.964006 , 0.961019 , 0.951786 , 0.957457 , 0.950327 , 0.954375 , 0.9608 , 0.965864 , 0.982396 , 1.011334 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.289798 , 1.126375 , 1.074621 , 1.069365 , 1.051364 , 1.009432 , 0.984578 , 0.991187 , 1.016306 , 1.083681 , 1.076539 , 1.104657 , 1.114005 , 1.081102 , 1.039196 , 1.009524 , 0.966515 , 0.953688 , 0.966274 , 0.973232 , 0.966402 , 0.946801 , 0.951952 , 0.965679 , 0.96977 , 0.946541 , 0.941678 , 0.937528 , 0.922351 , 0.914192 , 0.92879 , 0.932284 , 0.933182 , 0.922322 , 0.91851 , 0.919591 , 0.925027 , 0.924611 , 0.932288 , 0.933352 , 0.930517 , 0.931666 , 0.931763 , 0.932655 , 0.928945 , 0.933308 , 0.932392 , 0.932943 , 0.935328 , 0.947019 , 0.954093 , 0.95156 , 0.939591 , 0.942808 , 0.944862 , 0.944004 , 0.949161 , 0.950992 , 0.956738 , 0.951184 , 0.953545 , 0.958836 , 0.966134 , 0.956752 , 0.951961 , 0.958667 , 0.9579 , 0.968531 , 0.973792 , 0.969238 , 0.970838 , 0.954552 , 0.968166 , 0.989176 , 0.974784 , 0.970674 , 0.9733 , 0.990576 , 1.0062 , 1.010295 , 0.99378 , 0.986109 , 1.007054 , 1.005377 , 1.010013 , 1.014671 , 1.021618 , 1.021229 , 1.021003 , 1.020866 , 1.029358 , 1.042136 , 1.030482 , 1.019556 , 1.036656 , 1.05348 , 1.015947 , 1.07263 , 1.092879 , 1.053624 , 1.086491 , 1.139334 , 1.163645 , 1.162487 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 }
wavelength = { 346.6325373 , 351.6412073 , 356.6498773 , 361.6585473 , 366.6672173 , 371.6758873 , 376.6845573 , 381.6932273 , 386.7018973 , 391.7105673 , 396.7192373 , 401.7279073 , 406.7365773 , 411.7452473 , 416.7539173 , 421.7625873 , 426.7712573 , 431.7799273 , 436.7885973 , 441.7972673 , 446.8059373 , 451.8146073 , 456.8232773 , 461.8319473 , 466.8406173 , 471.8492873 , 476.8579573 , 481.8666273 , 486.8752973 , 491.8839673 , 496.8926373 , 501.9013073 , 506.9099773 , 511.9186473 , 516.9273173 , 521.9359873 , 526.9446573 , 531.9533273 , 536.9619973 , 541.9706673 , 546.9793373 , 551.9880073 , 556.9966773 , 562.0053473 , 567.0140173 , 572.0226873 , 577.0313573 , 582.0400273 , 587.0486973 , 592.0573673 , 597.0660373 , 602.0747073 , 607.0833773 , 612.0920473 , 617.1007173 , 622.1093873 , 627.1180573 , 632.1267273 , 637.1353973 , 642.1440673 , 647.1527373 , 652.1614073 , 657.1700773 , 662.1787473 , 667.1874173 , 672.1960873 , 677.2047573 , 682.2134273 , 687.2220973 , 692.2307673 , 697.2394373 , 702.2481073 , 707.2567773 , 712.2654473 , 717.2741173 , 722.2827873 , 727.2914573 , 732.3001273 , 737.3087973 , 742.3174673 , 747.3261373 , 752.3348073 , 757.3434773 , 762.3521473 , 767.3608173 , 772.3694873 , 777.3781573 , 782.3868273 , 787.3954973 , 792.4041673 , 797.4128373 , 802.4215073 , 807.4301773 , 812.4388473 , 817.4475173 , 822.4561873 , 827.4648573 , 832.4735273 , 837.4821973 , 842.4908673 , 847.4995373 , 852.5082073 , 857.5168773 , 862.5255473 , 867.5342173 , 872.5428873 , 877.5515573 , 882.5602273 , 887.5688973 , 892.5775673 , 897.5862373 , 902.5949073 , 907.6035773 , 912.6122473 , 917.6209173 , 922.6295873 , 927.6382573 , 932.6469273 , 937.6555973 , 942.6642673 , 947.6729373 , 952.6816073 , 957.6902773 , 962.6989473 , 967.7076173 , 972.7162873 , 977.7249573 , 982.7336273 , 987.7422973 , 992.7509673 , 997.7596373 , 1002.7683073 , 1007.7769773 , 1012.7856473 , 1017.7943173 , 1022.8029873 , 1027.8116573 , 1032.8203273 , 1037.8289973 , 1042.8376673 , 1047.8463373 , 1052.8550073 , 1057.8636773 , 1062.8723473 , 1067.8810173 , 1072.8896873 , 1077.8983573 , 1082.9070373 , 1087.9157073 , 1092.9243673 , 1097.9330373 , 1102.9417073 , 1107.9503873 , 1112.9590473 , 1117.9677273 , 1122.9763873 , 1127.9850573 , 1132.9937273 , 1138.0024073 , 1143.0110673 , 1148.0197373 , 1153.0284073 , 1158.0370873 , 1163.0457473 , 1168.0544273 , 1173.0630973 , 1178.0717673 , 1183.0804373 , 1188.0891073 , 1193.0977773 , 1198.1064473 , 1203.1151173 , 1208.1237873 , 1213.1324573 , 1218.1411273 , 1223.1497973 , 1228.1584673 , 1233.1671373 , 1238.1758073 , 1243.1844773 , 1248.1931473 , 1253.2018173 , 1258.2104873 , 1263.2191573 , 1268.2278273 , 1273.2364973 , 1278.2451673 , 1283.2538373 , 1288.2625073 , 1293.2711773 , 1298.2798473 , 1303.2885173 , 1308.2971873 , 1313.3058573 , 1318.3145273 , 1323.3231973 , 1328.3318673 , 1333.3405373 , 1338.3492073 , 1343.3578773 , 1348.3665473 , 1353.3752173 , 1358.3838873 , 1363.3925573 , 1368.4012273 , 1373.4098973 , 1378.4185673 , 1383.4272373 , 1388.4359073 , 1393.4445773 , 1398.4532473 , 1403.4619173 , 1408.4705873 , 1413.4792573 , 1418.4879273 , 1423.4965973 , 1428.5052673 , 1433.5139373 , 1438.5226073 , 1443.5312773 , 1448.5399473 , 1453.5486173 , 1458.5572873 , 1463.5659573 , 1468.5746273 , 1473.5832973 , 1478.5919673 , 1483.6006373 , 1488.6093073 , 1493.6179773 , 1498.6266473 , 1503.6353173 , 1508.6439873 , 1513.6526573 , 1518.6613273 , 1523.6699973 , 1528.6786673 , 1533.6873373 , 1538.6960073 , 1543.7046773 , 1548.7133473 , 1553.7220173 , 1558.7306873 , 1563.7393573 , 1568.7480273 , 1573.7566973 , 1578.7653673 , 1583.7740373 , 1588.7827073 , 1593.7913773 , 1598.8000473 , 1603.8087173 , 1608.8173873 , 1613.8260573 , 1618.8347273 , 1623.8433973 , 1628.8520673 , 1633.8607373 , 1638.8694073 , 1643.8780773 , 1648.8867473 , 1653.8954173 , 1658.9040873 , 1663.9127573 , 1668.9214273 , 1673.9300973 , 1678.9387673 , 1683.9474373 , 1688.9561073 , 1693.9647773 , 1698.9734473 , 1703.9821173 , 1708.9907873 , 1713.9994573 , 1719.0081273 , 1724.0167973 , 1729.0254673 , 1734.0341373 , 1739.0428073 , 1744.0514773 , 1749.0601473 , 1754.0688173 , 1759.0774873 , 1764.0861573 , 1769.0948273 , 1774.1034973 , 1779.1121673 , 1784.1208373 , 1789.1295073 , 1794.1381773 , 1799.1468473 , 1804.1555173 , 1809.1641873 , 1814.1728573 , 1819.1815273 , 1824.1901973 , 1829.1988673 , 1834.2075373 , 1839.2162073 , 1844.2248773 , 1849.2335473 , 1854.2422173 , 1859.2508873 , 1864.2595573 , 1869.2682273 , 1874.2768973 , 1879.2855673 , 1884.2942373 , 1889.3029073 , 1894.3115773 , 1899.3202473 , 1904.3289173 , 1909.3375873 , 1914.3462573 , 1919.3549373 , 1924.3635973 , 1929.3722773 , 1934.3809373 , 1939.3896073 , 1944.3982873 , 1949.4069573 , 1954.4156173 , 1959.4242873 , 1964.4329673 , 1969.4416373 , 1974.4502973 , 1979.4589673 , 1984.4676373 , 1989.4763173 , 1994.4849773 , 1999.4936473 , 2004.5023273 , 2009.5109973 , 2014.5196673 , 2019.5283373 , 2024.5370073 , 2029.5456773 , 2034.5543473 , 2039.5630173 , 2044.5716873 , 2049.5803573 , 2054.5890273 , 2059.5976973 , 2064.6063673 , 2069.6150373 , 2074.6237073 , 2079.6323773 , 2084.6410473 , 2089.6497173 , 2094.6583873 , 2099.6670573 , 2104.6757273 , 2109.6843973 , 2114.6930673 , 2119.7017373 , 2124.7104073 , 2129.7190773 , 2134.7277473 , 2139.7364173 , 2144.7450873 , 2149.7537573 , 2154.7624273 , 2159.7710973 , 2164.7797673 , 2169.7884373 , 2174.7971073 , 2179.8057773 , 2184.8144473 , 2189.8231173 , 2194.8317873 , 2199.8404573 , 2204.8491273 , 2209.8577973 , 2214.8664673 , 2219.8751373 , 2224.8838073 , 2229.8924773 , 2234.9011473 , 2239.9098173 , 2244.9184873 , 2249.9271573 , 2254.9358273 , 2259.9444973 , 2264.9531673 , 2269.9618373 , 2274.9705073 , 2279.9791773 , 2284.9878473 , 2289.9965173 , 2295.0051873 , 2300.0138573 , 2305.0225273 , 2310.0311973 , 2315.0398673 , 2320.0485373 , 2325.0572073 , 2330.0658773 , 2335.0745473 , 2340.0832173 , 2345.0918873 , 2350.1005573 , 2355.1092273 , 2360.1178973 , 2365.1265673 , 2370.1352373 , 2375.1439073 , 2380.1525773 , 2385.1612473 , 2390.1699173 , 2395.1785873 , 2400.1872573 , 2405.1959273 , 2410.2045973 , 2415.2132673 , 2420.2219373 , 2425.2306073 , 2430.2392773 , 2435.2479473 , 2440.2566173 , 2445.2652873 , 2450.2739573 , 2455.2826273 , 2460.2912973 , 2465.2999673 , 2470.3086373 , 2475.3173073 , 2480.3259773 , 2485.3346473 , 2490.3433173 , 2495.3519873 , 2500.3606573 , 2505.3693273 }
fwhm = { 5.55165 , 5.55524 , 5.55879 , 5.5623 , 5.56577 , 5.5692 , 5.5726 , 5.57595 , 5.57927 , 5.58255 , 5.58579 , 5.58899 , 5.59216 , 5.59529 , 5.59839 , 5.60144 , 5.60446 , 5.60745 , 5.6104 , 5.61332 , 5.61619 , 5.61904 , 5.62185 , 5.62463 , 5.62737 , 5.63008 , 5.63275 , 5.6354 , 5.63801 , 5.64058 , 5.64313 , 5.64564 , 5.64812 , 5.65057 , 5.65299 , 5.65537 , 5.65773 , 5.66005 , 5.66235 , 5.66461 , 5.66684 , 5.66905 , 5.67122 , 5.67337 , 5.67549 , 5.67758 , 5.67964 , 5.68167 , 5.68367 , 5.68565 , 5.6876 , 5.68952 , 5.69141 , 5.69328 , 5.69512 , 5.69694 , 5.69873 , 5.70049 , 5.70223 , 5.70394 , 5.70563 , 5.70729 , 5.70893 , 5.71055 , 5.71214 , 5.7137 , 5.71524 , 5.71676 , 5.71826 , 5.71973 , 5.72118 , 5.72261 , 5.72401 , 5.72539 , 5.72676 , 5.72809 , 5.72941 , 5.73071 , 5.73198 , 5.73324 , 5.73447 , 5.73569 , 5.73688 , 5.73806 , 5.73921 , 5.74034 , 5.74146 , 5.74256 , 5.74363 , 5.74469 , 5.74573 , 5.74676 , 5.74776 , 5.74875 , 5.74972 , 5.75067 , 5.7516 , 5.75252 , 5.75342 , 5.75431 , 5.75518 , 5.75603 , 5.75687 , 5.75769 , 5.75849 , 5.75928 , 5.76006 , 5.76082 , 5.76156 , 5.76229 , 5.76301 , 5.76371 , 5.7644 , 5.76507 , 5.76573 , 5.76638 , 5.76702 , 5.76764 , 5.76825 , 5.76884 , 5.76943 , 5.77 , 5.77056 , 5.77111 , 5.77164 , 5.77217 , 5.77268 , 5.77319 , 5.77368 , 5.77416 , 5.77463 , 5.77509 , 5.77554 , 5.77598 , 5.77641 , 5.77683 , 5.77725 , 5.77765 , 5.77804 , 5.77843 , 5.7788 , 5.77917 , 5.77953 , 5.77988 , 5.78022 , 5.78056 , 5.78088 , 5.7812 , 5.78151 , 5.78182 , 5.78212 , 5.78241 , 5.78269 , 5.78297 , 5.78324 , 5.78351 , 5.78377 , 5.78402 , 5.78427 , 5.78452 , 5.78475 , 5.78499 , 5.78521 , 5.78543 , 5.78565 , 5.78587 , 5.78607 , 5.78628 , 5.78648 , 5.78668 , 5.78687 , 5.78706 , 5.78724 , 5.78743 , 5.78761 , 5.78778 , 5.78796 , 5.78813 , 5.7883 , 5.78846 , 5.78862 , 5.78879 , 5.78895 , 5.7891 , 5.78926 , 5.78941 , 5.78957 , 5.78972 , 5.78987 , 5.79002 , 5.79017 , 5.79032 , 5.79046 , 5.79061 , 5.79076 , 5.79091 , 5.79105 , 5.7912 , 5.79135 , 5.7915 , 5.79164 , 5.79179 , 5.79194 , 5.79209 , 5.79224 , 5.7924 , 5.79255 , 5.79271 , 5.79286 , 5.79302 , 5.79318 , 5.79335 , 5.79351 , 5.79368 , 5.79385 , 5.79402 , 5.79419 , 5.79437 , 5.79455 , 5.79473 , 5.79492 , 5.79511 , 5.7953 , 5.7955 , 5.7957 , 5.7959 , 5.7961 , 5.79631 , 5.79653 , 5.79675 , 5.79697 , 5.7972 , 5.79743 , 5.79766 , 5.7979 , 5.79815 , 5.7984 , 5.79865 , 5.79891 , 5.79918 , 5.79945 , 5.79972 , 5.80001 , 5.80029 , 5.80058 , 5.80088 , 5.80119 , 5.8015 , 5.80181 , 5.80213 , 5.80246 , 5.8028 , 5.80314 , 5.80349 , 5.80384 , 5.8042 , 5.80457 , 5.80494 , 5.80532 , 5.80571 , 5.80611 , 5.80651 , 5.80692 , 5.80734 , 5.80777 , 5.8082 , 5.80864 , 5.80909 , 5.80955 , 5.81001 , 5.81048 , 5.81096 , 5.81145 , 5.81195 , 5.81246 , 5.81297 , 5.81349 , 5.81402 , 5.81456 , 5.81511 , 5.81567 , 5.81623 , 5.81681 , 5.81739 , 5.81799 , 5.81859 , 5.8192 , 5.81982 , 5.82045 , 5.82109 , 5.82174 , 5.8224 , 5.82307 , 5.82374 , 5.82443 , 5.82513 , 5.82584 , 5.82656 , 5.82728 , 5.82802 , 5.82877 , 5.82953 , 5.8303 , 5.83108 , 5.83186 , 5.83266 , 5.83347 , 5.8343 , 5.83513 , 5.83597 , 5.83682 , 5.83769 , 5.83856 , 5.83944 , 5.84034 , 5.84125 , 5.84217 , 5.8431 , 5.84404 , 5.84499 , 5.84595 , 5.84692 , 5.84791 , 5.84891 , 5.84992 , 5.85093 , 5.85197 , 5.85301 , 5.85406 , 5.85513 , 5.85621 , 5.85729 , 5.8584 , 5.85951 , 5.86063 , 5.86177 , 5.86292 , 5.86408 , 5.86525 , 5.86643 , 5.86763 , 5.86884 , 5.87005 , 5.87129 , 5.87253 , 5.87379 , 5.87506 , 5.87634 , 5.87763 , 5.87893 , 5.88025 , 5.88158 , 5.88292 , 5.88428 , 5.88564 , 5.88702 , 5.88841 , 5.88982 , 5.89123 , 5.89266 , 5.8941 , 5.89556 , 5.89702 , 5.8985 , 5.89999 , 5.90149 , 5.90301 , 5.90454 , 5.90608 , 5.90763 , 5.9092 , 5.91078 , 5.91237 , 5.91398 , 5.91559 , 5.91722 , 5.91886 , 5.92052 , 5.92219 , 5.92387 , 5.92556 , 5.92727 , 5.92898 , 5.93071 , 5.93246 , 5.93421 , 5.93598 , 5.93776 , 5.93956 , 5.94137 , 5.94319 , 5.94502 , 5.94686 , 5.94872 , 5.95059 , 5.95247 , 5.95437 , 5.95628 , 5.9582 , 5.96013 , 5.96207 , 5.96403 , 5.966 , 5.96799 , 5.96998 , 5.97199 , 5.97401 , 5.97604 , 5.97809 , 5.98015 , 5.98222 , 5.9843 , 5.98639 , 5.9885 , 5.99062 , 5.99275 , 5.9949 , 5.99706 , 5.99922 , 6.00141 , 6.0036 , 6.0058 , 6.00802 , 6.01025 , 6.01249 , 6.01475 , 6.01701 , 6.01929 , 6.02158 , 6.02388 , 6.02619 , 6.02852 }
bbl = { 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 }
smoothing factors = { 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 }

]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [Content-Type] with value [text/plain; charset=ISO-8859-1]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added config file met key [ProductType] with value [GenericFile]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.crawl.ProductCrawler ingest
INFO: ProductCrawler: Ready to ingest product: [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr]: ProductType: [GenericFile]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.filemgr.ingest.StdIngester setFileManager
INFO: StdIngester: connected to file manager: [http://localhost:9000]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferer setFileManagerUrl
INFO: In Place Data Transfer to: [http://localhost:9000] enabled
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.filemgr.ingest.StdIngester ingest
INFO: StdIngester: ingesting product: ProductName: [ang20150419t155032_corr_v1f_img.hdr]: ProductType: [GenericFile]: FileLocation: [/usr/local/coal-sds-deploy/data/staging/]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsFlat
FINE: VersioningUtils: Generated data store ref: file:/usr/local/coal-sds-deploy/data/archive/ang20150419t155032_corr_v1f_img.hdr/ang20150419t155032_corr_v1f_img.hdr from origRef: file:/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.crawl.ProductCrawler ingest
INFO: Successfully ingested product: [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr]: product id: c3b4484f-4f7b-4b77-a6db-0bd20b3e117e
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
INFO: Successful ingest of product: [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
INFO: Handling file /usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr.met
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Invoking tika extractor on file [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr.met]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Number of captured tika metadata keys: [4]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [X-Parsed-By] with value [org.apache.tika.parser.DefaultParser]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [Content-Encoding] with value [ISO-8859-1]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [content] with value [<cas:metadata xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
</cas:metadata>
]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added tika met key [Content-Type] with value [text/plain; charset=ISO-8859-1]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor extrMetadata
FINE: Added config file met key [ProductType] with value [GenericFile]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.crawl.ProductCrawler ingest
INFO: ProductCrawler: Ready to ingest product: [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr.met]: ProductType: [GenericFile]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.filemgr.ingest.StdIngester ingest
INFO: StdIngester: ingesting product: ProductName: [ang20150419t155032_corr_v1f_img.hdr.met]: ProductType: [GenericFile]: FileLocation: [/usr/local/coal-sds-deploy/data/staging/]
Sep 28, 2018 2:31:32 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsFlat
FINE: VersioningUtils: Generated data store ref: file:/usr/local/coal-sds-deploy/data/archive/ang20150419t155032_corr_v1f_img.hdr.met/ang20150419t155032_corr_v1f_img.hdr.met from origRef: file:/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr.met
Sep 28, 2018 2:31:33 PM org.apache.oodt.cas.crawl.ProductCrawler ingest
INFO: Successfully ingested product: [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr.met]: product id: 275d2ec2-0c95-453f-8a3f-661cf1a40e98
Sep 28, 2018 2:31:33 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
INFO: Successful ingest of product: [/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr.met]

The product is then ingested and looks as follows

      {
        "id": "c3b4484f-4f7b-4b77-a6db-0bd20b3e117e",
        "CAS.ProductStructure": "Flat",
        "CAS.ProductTypeName": "GenericFile",
        "CAS.ProductName": "ang20150419t155032_corr_v1f_img.hdr",
        "CAS.ProductReceivedTime": "2018-09-28T14:31:32Z",
        "CAS.ProductTypeId": "urn:oodt:GenericFile",
        "CAS.ProductTransferStatus": "RECEIVED",
        "CAS.ProductId": "c3b4484f-4f7b-4b77-a6db-0bd20b3e117e",
        "X-Parsed-By": [
          "org.apache.tika.parser.DefaultParser"
        ],
        "ProductName": [
          "ang20150419t155032_corr_v1f_img.hdr"
        ],
        "FileLocation": [
          "/usr/local/coal-sds-deploy/data/archive/ang20150419t155032_corr_v1f_img.hdr"
        ],
        "Content-Encoding": [
          "ISO-8859-1"
        ],
        "Filename": [
          "ang20150419t155032_corr_v1f_img.hdr"
        ],
        "content": [
          "ENVI\ndescription = {\n  Georeferenced Image built from input GLT. [Tue Aug 11 15:39:46 2015] [Tue\n  Aug 11 16:17:51 2015]}\nsamples = 1363\nlines = 22826\nbands = 432\nheader offset = 0\nfile type = ENVI Standard\ndata type = 4\ninterleave = bil\nsensor type = Unknown\nbyte order = 0\nmap info = { UTM , 1.000 , 1.000 , 201603.137 , 4061363.983 , 2.7000000000e+00 , 2.7000000000e+00 , 13 , North , WGS-84 , units=Meters , rotation=90.00000000 }\nwavelength units = Nanometers\ncorrection factors = { 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.880586 , 1.741631 , 1.524632 , 1.564904 , 1.390035 , 1.342501 , 1.264768 , 1.200357 , 1.131471 , 1.06512 , 1.105211 , 1.081234 , 1.064999 , 1.056798 , 1.042573 , 1.009621 , 1.009031 , 1.002107 , 0.997332 , 0.976807 , 0.963755 , 0.969028 , 0.962823 , 0.949522 , 0.960568 , 0.957813 , 0.934766 , 0.944994 , 0.937726 , 0.935257 , 0.932706 , 0.932568 , 0.933217 , 0.928705 , 0.929294 , 0.936669 , 0.935498 , 0.94823 , 0.949846 , 0.945885 , 0.935468 , 0.930084 , 0.934473 , 0.935378 , 0.939193 , 0.935081 , 0.937398 , 0.943396 , 0.947133 , 0.950645 , 0.945531 , 0.940295 , 0.933129 , 0.930664 , 0.92736 , 0.931786 , 0.928217 , 0.928205 , 0.926481 , 0.928583 , 0.930504 , 0.93648 , 0.930731 , 0.931265 , 0.935063 , 0.93434 , 0.926983 , 0.932689 , 0.936477 , 0.939647 , 0.940155 , 0.937519 , 0.939448 , 0.942124 , 0.93653 , 0.9435 , 0.959204 , 0.942566 , 0.940873 , 0.939414 , 0.939822 , 0.940174 , 0.941372 , 0.939347 , 0.942108 , 0.942664 , 0.934811 , 0.934567 , 0.937712 , 0.940611 , 0.944809 , 0.939877 , 0.943376 , 0.939189 , 0.943619 , 0.946268 , 0.940166 , 0.953752 , 0.958975 , 0.954512 , 0.954103 , 0.958978 , 0.953247 , 0.952199 , 0.956082 , 0.957846 , 0.970078 , 0.973704 , 0.980014 , 0.928845 , 0.922973 , 0.954414 , 0.95521 , 0.961276 , 0.964513 , 0.965296 , 0.964644 , 0.954999 , 0.951133 , 0.956216 , 0.951977 , 0.948547 , 0.949499 , 0.952685 , 0.950158 , 0.944263 , 0.936946 , 0.938394 , 0.941325 , 0.94116 , 0.941397 , 0.940811 , 0.942695 , 0.945228 , 0.953929 , 0.962457 , 0.968728 , 0.963947 , 0.961222 , 0.963003 , 0.967658 , 0.969773 , 0.970294 , 0.963456 , 0.970497 , 0.976972 , 0.961611 , 0.953081 , 0.945668 , 0.993867 , 1.019915 , 0.997013 , 0.977643 , 0.998022 , 1.007041 , 1.003881 , 0.991335 , 0.976202 , 0.967636 , 0.969294 , 0.965331 , 0.968705 , 0.965705 , 0.973601 , 0.97282 , 0.970848 , 0.970687 , 0.969394 , 0.972263 , 0.969286 , 0.970327 , 0.97754 , 0.984703 , 0.993916 , 1.02186 , 1.054704 , 1.061183 , 1.036962 , 1.012519 , 0.991209 , 0.975974 , 0.965446 , 0.958801 , 0.951519 , 0.960628 , 0.957276 , 0.96061 , 0.95689 , 0.956666 , 0.965567 , 0.982251 , 1.038526 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.131643 , 1.150333 , 1.112466 , 1.06255 , 1.033182 , 1.03018 , 1.006512 , 0.993285 , 1.003557 , 1.014441 , 0.998586 , 0.994426 , 0.987975 , 0.984681 , 0.981863 , 0.964106 , 0.956432 , 0.954467 , 0.956429 , 0.950668 , 0.947217 , 0.944635 , 0.942942 , 0.941078 , 0.943502 , 0.950408 , 0.957625 , 0.965073 , 0.976012 , 0.976683 , 0.975625 , 0.968011 , 0.968843 , 0.970632 , 0.960977 , 0.960505 , 0.955015 , 0.953597 , 0.951119 , 0.945679 , 0.949988 , 0.951236 , 0.947813 , 0.948004 , 0.950015 , 0.939258 , 0.945863 , 0.953927 , 0.953145 , 0.945291 , 0.942319 , 0.947022 , 0.948264 , 0.947112 , 0.942092 , 0.943128 , 0.948068 , 0.944432 , 0.950396 , 0.964006 , 0.961019 , 0.951786 , 0.957457 , 0.950327 , 0.954375 , 0.9608 , 0.965864 , 0.982396 , 1.011334 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.289798 , 1.126375 , 1.074621 , 1.069365 , 1.051364 , 1.009432 , 0.984578 , 0.991187 , 1.016306 , 1.083681 , 1.076539 , 1.104657 , 1.114005 , 1.081102 , 1.039196 , 1.009524 , 0.966515 , 0.953688 , 0.966274 , 0.973232 , 0.966402 , 0.946801 , 0.951952 , 0.965679 , 0.96977 , 0.946541 , 0.941678 , 0.937528 , 0.922351 , 0.914192 , 0.92879 , 0.932284 , 0.933182 , 0.922322 , 0.91851 , 0.919591 , 0.925027 , 0.924611 , 0.932288 , 0.933352 , 0.930517 , 0.931666 , 0.931763 , 0.932655 , 0.928945 , 0.933308 , 0.932392 , 0.932943 , 0.935328 , 0.947019 , 0.954093 , 0.95156 , 0.939591 , 0.942808 , 0.944862 , 0.944004 , 0.949161 , 0.950992 , 0.956738 , 0.951184 , 0.953545 , 0.958836 , 0.966134 , 0.956752 , 0.951961 , 0.958667 , 0.9579 , 0.968531 , 0.973792 , 0.969238 , 0.970838 , 0.954552 , 0.968166 , 0.989176 , 0.974784 , 0.970674 , 0.9733 , 0.990576 , 1.0062 , 1.010295 , 0.99378 , 0.986109 , 1.007054 , 1.005377 , 1.010013 , 1.014671 , 1.021618 , 1.021229 , 1.021003 , 1.020866 , 1.029358 , 1.042136 , 1.030482 , 1.019556 , 1.036656 , 1.05348 , 1.015947 , 1.07263 , 1.092879 , 1.053624 , 1.086491 , 1.139334 , 1.163645 , 1.162487 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 }\nwavelength = { 346.6325373 , 351.6412073 , 356.6498773 , 361.6585473 , 366.6672173 , 371.6758873 , 376.6845573 , 381.6932273 , 386.7018973 , 391.7105673 , 396.7192373 , 401.7279073 , 406.7365773 , 411.7452473 , 416.7539173 , 421.7625873 , 426.7712573 , 431.7799273 , 436.7885973 , 441.7972673 , 446.8059373 , 451.8146073 , 456.8232773 , 461.8319473 , 466.8406173 , 471.8492873 , 476.8579573 , 481.8666273 , 486.8752973 , 491.8839673 , 496.8926373 , 501.9013073 , 506.9099773 , 511.9186473 , 516.9273173 , 521.9359873 , 526.9446573 , 531.9533273 , 536.9619973 , 541.9706673 , 546.9793373 , 551.9880073 , 556.9966773 , 562.0053473 , 567.0140173 , 572.0226873 , 577.0313573 , 582.0400273 , 587.0486973 , 592.0573673 , 597.0660373 , 602.0747073 , 607.0833773 , 612.0920473 , 617.1007173 , 622.1093873 , 627.1180573 , 632.1267273 , 637.1353973 , 642.1440673 , 647.1527373 , 652.1614073 , 657.1700773 , 662.1787473 , 667.1874173 , 672.1960873 , 677.2047573 , 682.2134273 , 687.2220973 , 692.2307673 , 697.2394373 , 702.2481073 , 707.2567773 , 712.2654473 , 717.2741173 , 722.2827873 , 727.2914573 , 732.3001273 , 737.3087973 , 742.3174673 , 747.3261373 , 752.3348073 , 757.3434773 , 762.3521473 , 767.3608173 , 772.3694873 , 777.3781573 , 782.3868273 , 787.3954973 , 792.4041673 , 797.4128373 , 802.4215073 , 807.4301773 , 812.4388473 , 817.4475173 , 822.4561873 , 827.4648573 , 832.4735273 , 837.4821973 , 842.4908673 , 847.4995373 , 852.5082073 , 857.5168773 , 862.5255473 , 867.5342173 , 872.5428873 , 877.5515573 , 882.5602273 , 887.5688973 , 892.5775673 , 897.5862373 , 902.5949073 , 907.6035773 , 912.6122473 , 917.6209173 , 922.6295873 , 927.6382573 , 932.6469273 , 937.6555973 , 942.6642673 , 947.6729373 , 952.6816073 , 957.6902773 , 962.6989473 , 967.7076173 , 972.7162873 , 977.7249573 , 982.7336273 , 987.7422973 , 992.7509673 , 997.7596373 , 1002.7683073 , 1007.7769773 , 1012.7856473 , 1017.7943173 , 1022.8029873 , 1027.8116573 , 1032.8203273 , 1037.8289973 , 1042.8376673 , 1047.8463373 , 1052.8550073 , 1057.8636773 , 1062.8723473 , 1067.8810173 , 1072.8896873 , 1077.8983573 , 1082.9070373 , 1087.9157073 , 1092.9243673 , 1097.9330373 , 1102.9417073 , 1107.9503873 , 1112.9590473 , 1117.9677273 , 1122.9763873 , 1127.9850573 , 1132.9937273 , 1138.0024073 , 1143.0110673 , 1148.0197373 , 1153.0284073 , 1158.0370873 , 1163.0457473 , 1168.0544273 , 1173.0630973 , 1178.0717673 , 1183.0804373 , 1188.0891073 , 1193.0977773 , 1198.1064473 , 1203.1151173 , 1208.1237873 , 1213.1324573 , 1218.1411273 , 1223.1497973 , 1228.1584673 , 1233.1671373 , 1238.1758073 , 1243.1844773 , 1248.1931473 , 1253.2018173 , 1258.2104873 , 1263.2191573 , 1268.2278273 , 1273.2364973 , 1278.2451673 , 1283.2538373 , 1288.2625073 , 1293.2711773 , 1298.2798473 , 1303.2885173 , 1308.2971873 , 1313.3058573 , 1318.3145273 , 1323.3231973 , 1328.3318673 , 1333.3405373 , 1338.3492073 , 1343.3578773 , 1348.3665473 , 1353.3752173 , 1358.3838873 , 1363.3925573 , 1368.4012273 , 1373.4098973 , 1378.4185673 , 1383.4272373 , 1388.4359073 , 1393.4445773 , 1398.4532473 , 1403.4619173 , 1408.4705873 , 1413.4792573 , 1418.4879273 , 1423.4965973 , 1428.5052673 , 1433.5139373 , 1438.5226073 , 1443.5312773 , 1448.5399473 , 1453.5486173 , 1458.5572873 , 1463.5659573 , 1468.5746273 , 1473.5832973 , 1478.5919673 , 1483.6006373 , 1488.6093073 , 1493.6179773 , 1498.6266473 , 1503.6353173 , 1508.6439873 , 1513.6526573 , 1518.6613273 , 1523.6699973 , 1528.6786673 , 1533.6873373 , 1538.6960073 , 1543.7046773 , 1548.7133473 , 1553.7220173 , 1558.7306873 , 1563.7393573 , 1568.7480273 , 1573.7566973 , 1578.7653673 , 1583.7740373 , 1588.7827073 , 1593.7913773 , 1598.8000473 , 1603.8087173 , 1608.8173873 , 1613.8260573 , 1618.8347273 , 1623.8433973 , 1628.8520673 , 1633.8607373 , 1638.8694073 , 1643.8780773 , 1648.8867473 , 1653.8954173 , 1658.9040873 , 1663.9127573 , 1668.9214273 , 1673.9300973 , 1678.9387673 , 1683.9474373 , 1688.9561073 , 1693.9647773 , 1698.9734473 , 1703.9821173 , 1708.9907873 , 1713.9994573 , 1719.0081273 , 1724.0167973 , 1729.0254673 , 1734.0341373 , 1739.0428073 , 1744.0514773 , 1749.0601473 , 1754.0688173 , 1759.0774873 , 1764.0861573 , 1769.0948273 , 1774.1034973 , 1779.1121673 , 1784.1208373 , 1789.1295073 , 1794.1381773 , 1799.1468473 , 1804.1555173 , 1809.1641873 , 1814.1728573 , 1819.1815273 , 1824.1901973 , 1829.1988673 , 1834.2075373 , 1839.2162073 , 1844.2248773 , 1849.2335473 , 1854.2422173 , 1859.2508873 , 1864.2595573 , 1869.2682273 , 1874.2768973 , 1879.2855673 , 1884.2942373 , 1889.3029073 , 1894.3115773 , 1899.3202473 , 1904.3289173 , 1909.3375873 , 1914.3462573 , 1919.3549373 , 1924.3635973 , 1929.3722773 , 1934.3809373 , 1939.3896073 , 1944.3982873 , 1949.4069573 , 1954.4156173 , 1959.4242873 , 1964.4329673 , 1969.4416373 , 1974.4502973 , 1979.4589673 , 1984.4676373 , 1989.4763173 , 1994.4849773 , 1999.4936473 , 2004.5023273 , 2009.5109973 , 2014.5196673 , 2019.5283373 , 2024.5370073 , 2029.5456773 , 2034.5543473 , 2039.5630173 , 2044.5716873 , 2049.5803573 , 2054.5890273 , 2059.5976973 , 2064.6063673 , 2069.6150373 , 2074.6237073 , 2079.6323773 , 2084.6410473 , 2089.6497173 , 2094.6583873 , 2099.6670573 , 2104.6757273 , 2109.6843973 , 2114.6930673 , 2119.7017373 , 2124.7104073 , 2129.7190773 , 2134.7277473 , 2139.7364173 , 2144.7450873 , 2149.7537573 , 2154.7624273 , 2159.7710973 , 2164.7797673 , 2169.7884373 , 2174.7971073 , 2179.8057773 , 2184.8144473 , 2189.8231173 , 2194.8317873 , 2199.8404573 , 2204.8491273 , 2209.8577973 , 2214.8664673 , 2219.8751373 , 2224.8838073 , 2229.8924773 , 2234.9011473 , 2239.9098173 , 2244.9184873 , 2249.9271573 , 2254.9358273 , 2259.9444973 , 2264.9531673 , 2269.9618373 , 2274.9705073 , 2279.9791773 , 2284.9878473 , 2289.9965173 , 2295.0051873 , 2300.0138573 , 2305.0225273 , 2310.0311973 , 2315.0398673 , 2320.0485373 , 2325.0572073 , 2330.0658773 , 2335.0745473 , 2340.0832173 , 2345.0918873 , 2350.1005573 , 2355.1092273 , 2360.1178973 , 2365.1265673 , 2370.1352373 , 2375.1439073 , 2380.1525773 , 2385.1612473 , 2390.1699173 , 2395.1785873 , 2400.1872573 , 2405.1959273 , 2410.2045973 , 2415.2132673 , 2420.2219373 , 2425.2306073 , 2430.2392773 , 2435.2479473 , 2440.2566173 , 2445.2652873 , 2450.2739573 , 2455.2826273 , 2460.2912973 , 2465.2999673 , 2470.3086373 , 2475.3173073 , 2480.3259773 , 2485.3346473 , 2490.3433173 , 2495.3519873 , 2500.3606573 , 2505.3693273 }\nfwhm = { 5.55165 , 5.55524 , 5.55879 , 5.5623 , 5.56577 , 5.5692 , 5.5726 , 5.57595 , 5.57927 , 5.58255 , 5.58579 , 5.58899 , 5.59216 , 5.59529 , 5.59839 , 5.60144 , 5.60446 , 5.60745 , 5.6104 , 5.61332 , 5.61619 , 5.61904 , 5.62185 , 5.62463 , 5.62737 , 5.63008 , 5.63275 , 5.6354 , 5.63801 , 5.64058 , 5.64313 , 5.64564 , 5.64812 , 5.65057 , 5.65299 , 5.65537 , 5.65773 , 5.66005 , 5.66235 , 5.66461 , 5.66684 , 5.66905 , 5.67122 , 5.67337 , 5.67549 , 5.67758 , 5.67964 , 5.68167 , 5.68367 , 5.68565 , 5.6876 , 5.68952 , 5.69141 , 5.69328 , 5.69512 , 5.69694 , 5.69873 , 5.70049 , 5.70223 , 5.70394 , 5.70563 , 5.70729 , 5.70893 , 5.71055 , 5.71214 , 5.7137 , 5.71524 , 5.71676 , 5.71826 , 5.71973 , 5.72118 , 5.72261 , 5.72401 , 5.72539 , 5.72676 , 5.72809 , 5.72941 , 5.73071 , 5.73198 , 5.73324 , 5.73447 , 5.73569 , 5.73688 , 5.73806 , 5.73921 , 5.74034 , 5.74146 , 5.74256 , 5.74363 , 5.74469 , 5.74573 , 5.74676 , 5.74776 , 5.74875 , 5.74972 , 5.75067 , 5.7516 , 5.75252 , 5.75342 , 5.75431 , 5.75518 , 5.75603 , 5.75687 , 5.75769 , 5.75849 , 5.75928 , 5.76006 , 5.76082 , 5.76156 , 5.76229 , 5.76301 , 5.76371 , 5.7644 , 5.76507 , 5.76573 , 5.76638 , 5.76702 , 5.76764 , 5.76825 , 5.76884 , 5.76943 , 5.77 , 5.77056 , 5.77111 , 5.77164 , 5.77217 , 5.77268 , 5.77319 , 5.77368 , 5.77416 , 5.77463 , 5.77509 , 5.77554 , 5.77598 , 5.77641 , 5.77683 , 5.77725 , 5.77765 , 5.77804 , 5.77843 , 5.7788 , 5.77917 , 5.77953 , 5.77988 , 5.78022 , 5.78056 , 5.78088 , 5.7812 , 5.78151 , 5.78182 , 5.78212 , 5.78241 , 5.78269 , 5.78297 , 5.78324 , 5.78351 , 5.78377 , 5.78402 , 5.78427 , 5.78452 , 5.78475 , 5.78499 , 5.78521 , 5.78543 , 5.78565 , 5.78587 , 5.78607 , 5.78628 , 5.78648 , 5.78668 , 5.78687 , 5.78706 , 5.78724 , 5.78743 , 5.78761 , 5.78778 , 5.78796 , 5.78813 , 5.7883 , 5.78846 , 5.78862 , 5.78879 , 5.78895 , 5.7891 , 5.78926 , 5.78941 , 5.78957 , 5.78972 , 5.78987 , 5.79002 , 5.79017 , 5.79032 , 5.79046 , 5.79061 , 5.79076 , 5.79091 , 5.79105 , 5.7912 , 5.79135 , 5.7915 , 5.79164 , 5.79179 , 5.79194 , 5.79209 , 5.79224 , 5.7924 , 5.79255 , 5.79271 , 5.79286 , 5.79302 , 5.79318 , 5.79335 , 5.79351 , 5.79368 , 5.79385 , 5.79402 , 5.79419 , 5.79437 , 5.79455 , 5.79473 , 5.79492 , 5.79511 , 5.7953 , 5.7955 , 5.7957 , 5.7959 , 5.7961 , 5.79631 , 5.79653 , 5.79675 , 5.79697 , 5.7972 , 5.79743 , 5.79766 , 5.7979 , 5.79815 , 5.7984 , 5.79865 , 5.79891 , 5.79918 , 5.79945 , 5.79972 , 5.80001 , 5.80029 , 5.80058 , 5.80088 , 5.80119 , 5.8015 , 5.80181 , 5.80213 , 5.80246 , 5.8028 , 5.80314 , 5.80349 , 5.80384 , 5.8042 , 5.80457 , 5.80494 , 5.80532 , 5.80571 , 5.80611 , 5.80651 , 5.80692 , 5.80734 , 5.80777 , 5.8082 , 5.80864 , 5.80909 , 5.80955 , 5.81001 , 5.81048 , 5.81096 , 5.81145 , 5.81195 , 5.81246 , 5.81297 , 5.81349 , 5.81402 , 5.81456 , 5.81511 , 5.81567 , 5.81623 , 5.81681 , 5.81739 , 5.81799 , 5.81859 , 5.8192 , 5.81982 , 5.82045 , 5.82109 , 5.82174 , 5.8224 , 5.82307 , 5.82374 , 5.82443 , 5.82513 , 5.82584 , 5.82656 , 5.82728 , 5.82802 , 5.82877 , 5.82953 , 5.8303 , 5.83108 , 5.83186 , 5.83266 , 5.83347 , 5.8343 , 5.83513 , 5.83597 , 5.83682 , 5.83769 , 5.83856 , 5.83944 , 5.84034 , 5.84125 , 5.84217 , 5.8431 , 5.84404 , 5.84499 , 5.84595 , 5.84692 , 5.84791 , 5.84891 , 5.84992 , 5.85093 , 5.85197 , 5.85301 , 5.85406 , 5.85513 , 5.85621 , 5.85729 , 5.8584 , 5.85951 , 5.86063 , 5.86177 , 5.86292 , 5.86408 , 5.86525 , 5.86643 , 5.86763 , 5.86884 , 5.87005 , 5.87129 , 5.87253 , 5.87379 , 5.87506 , 5.87634 , 5.87763 , 5.87893 , 5.88025 , 5.88158 , 5.88292 , 5.88428 , 5.88564 , 5.88702 , 5.88841 , 5.88982 , 5.89123 , 5.89266 , 5.8941 , 5.89556 , 5.89702 , 5.8985 , 5.89999 , 5.90149 , 5.90301 , 5.90454 , 5.90608 , 5.90763 , 5.9092 , 5.91078 , 5.91237 , 5.91398 , 5.91559 , 5.91722 , 5.91886 , 5.92052 , 5.92219 , 5.92387 , 5.92556 , 5.92727 , 5.92898 , 5.93071 , 5.93246 , 5.93421 , 5.93598 , 5.93776 , 5.93956 , 5.94137 , 5.94319 , 5.94502 , 5.94686 , 5.94872 , 5.95059 , 5.95247 , 5.95437 , 5.95628 , 5.9582 , 5.96013 , 5.96207 , 5.96403 , 5.966 , 5.96799 , 5.96998 , 5.97199 , 5.97401 , 5.97604 , 5.97809 , 5.98015 , 5.98222 , 5.9843 , 5.98639 , 5.9885 , 5.99062 , 5.99275 , 5.9949 , 5.99706 , 5.99922 , 6.00141 , 6.0036 , 6.0058 , 6.00802 , 6.01025 , 6.01249 , 6.01475 , 6.01701 , 6.01929 , 6.02158 , 6.02388 , 6.02619 , 6.02852 }\nbbl = { 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 }\nsmoothing factors = { 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 , 1.0 }\n\n"
        ],
        "MimeType": [
          "application/octet-stream",
          "application",
          "octet-stream"
        ],
        "FileSize": [
          "20732"
        ],
        "Content-Type": [
          "text/plain; charset=ISO-8859-1"
        ],
        "CAS.ReferenceMimeType": [
          "application/octet-stream"
        ],
        "CAS.ReferenceDatastore": [
          "file:/usr/local/coal-sds-deploy/data/archive/ang20150419t155032_corr_v1f_img.hdr/ang20150419t155032_corr_v1f_img.hdr"
        ],
        "CAS.ReferenceFileSize": [
          20732
        ],
        "CAS.ReferenceOriginal": [
          "file:/usr/local/coal-sds-deploy/data/staging/ang20150419t155032_corr_v1f_img.hdr"
        ],
        "_version_": 1612888452999676000
      },

As you can see this is much better but it is not quite what we want for example

lewismc commented 6 years ago

We need to update to a newer version of Apache Tika in order to achieve the above. Right now we use Apache Tika versions

./crawler/lib/tika-core-1.10.jar
./crawler/lib/tika-parsers-1.7.jar

We should be using the most recent version 1.19

lewismc commented 6 years ago

This can be achieved as follows

./crawler_launcher 
  --filemgrUrl http://localhost:9000 
  --operation 
  --launchMetCrawler 
  --clientTransferer org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory 
  --productPath /usr/local/coal-sds-deploy/data/staging 
  --metExtractor org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor 
  --metExtractorConfig /usr/local/coal-sds-deploy/data/met/tika.conf 
  --failureDir /usr/local/coal-sds-deploy/data/failure/ 
  --daemonPort 9003 
  --daemonWait 2 
  --successDir /usr/local/coal-sds-deploy/data/archive/ 
  --actionIds DeleteDataFile

I would like someone to write this up so we will discuss this tomorrow.

lewismc commented 6 years ago

This now exists at https://github.com/capstone-coal/coal-sds/wiki/Automation-of-Data-Acquisition-and-Crawling

Thank you folks :)