USF-IMARS imars-etl issues

USF-IMARS / imars-etl

:cloud: Tools for `extract` and `load` for IMaRS ETL (Extract, Transform, Load) operations

0 stars 0 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

revisit

#48 7yl4r opened 6 months ago
0
breakout: filepath format/parse submodule(s)?

#47 7yl4r opened 4 years ago
0
breakout: arg validation submodule?

#46 7yl4r opened 4 years ago
0
check that file exists before loading

#45 7yl4r opened 4 years ago
0
breakout: db abstraction. rm airflow?

#44 7yl4r opened 4 years ago
2
product metadata DB fix spurrious status_id set to 3 & 5

#43 7yl4r closed 5 years ago
2
SyntaxError on `imars-etl select --first`

#42 7yl4r closed 5 years ago
0
TooManyMetadataMatchesException w/ == multihash are fine

#41 7yl4r closed 5 years ago
0
WV files with same `date_time`s

#40 7yl4r opened 5 years ago
13
implement test MetaDBHandler using filepath.formatter_hardcoded remnants

#39 7yl4r opened 5 years ago
0
find & re-download corrupted wv2 & wv3 .zip files

#38 7yl4r opened 5 years ago
6
imars_etl.load --update ?

#37 7yl4r opened 5 years ago
0
pymysql vs apache-airflow[mysql]

#36 7yl4r opened 5 years ago
0
ingest PGC WV* images

#35 7yl4r opened 5 years ago
3
How to expose (public?) metadata db to users?

#34 7yl4r opened 5 years ago
1
migrate data.py into the metadatadb

#33 7yl4r closed 5 years ago
2
auto-compression?

#32 7yl4r opened 5 years ago
2
imars_etl as hook(s)

#31 7yl4r opened 5 years ago
1
Load --dry_run actually loads object?

#30 7yl4r opened 5 years ago
0
set up ingest directory

#29 7yl4r opened 5 years ago
0
prevent --directory descent into product

#28 7yl4r closed 5 years ago
2
object_store-independent filepaths

#27 7yl4r closed 5 years ago
1
NASA CMR as metadata provider?

#26 7yl4r opened 5 years ago
0
rm duplicates from order list using wv* catalog ids

#25 7yl4r opened 5 years ago
0
clean & public

#24 7yl4r closed 4 years ago
5
attempted load appears to work. metadata never arrives.

#23 7yl4r closed 5 years ago
2
[Airflow-test server](ftp-ingest)Leading zeros are lost when parsing a file

#22 sebastiandig closed 5 years ago
1
fix wv2 & wv3 fmt strings

#21 7yl4r closed 6 years ago
0
check filepattern match before hashing when `load_dir`ing

#20 7yl4r closed 5 years ago
1
merge test in prep for af test sunday

#19 7yl4r closed 6 years ago
0
SQL injection protection

#18 7yl4r closed 5 years ago
1
reverse-load or count method?

#17 7yl4r closed 6 years ago
1
"drivers" as airflow hooks?

#16 7yl4r closed 5 years ago
4
+ wv2 xml metadata file driver

#15 7yl4r closed 6 years ago
2
filepath datetime parse not working well...

#14 7yl4r closed 5 years ago
4
filepath parsing could match multiple

#13 7yl4r opened 6 years ago
0
implement metadata file parsers

#12 7yl4r closed 6 years ago
0
+ default values to help

#11 7yl4r opened 6 years ago
0
restructure argument passing

#10 7yl4r closed 6 years ago
2
[metadata db] products loaded to wrong ids

#9 7yl4r closed 6 years ago
0
API load_dir w/o duplicates_ok does not raise

#8 7yl4r closed 5 years ago
1
implement real ORM

#7 7yl4r opened 6 years ago
7
+ `validate` subcommand

#6 7yl4r opened 6 years ago
0
replace usage of find regex in imars-dags

#5 7yl4r closed 6 years ago
1
imars-objects permissions

#4 7yl4r opened 6 years ago
4
improve imars_etl.filename.parse with parse module?

#3 7yl4r closed 6 years ago
0
filename parsing during `load` can rely a lot on `type`

#2 7yl4r closed 6 years ago
1
allow for insert values (area, type, etc) by shortName

#1 7yl4r closed 6 years ago
0