issues
search
Shopify
/
camus
Kafka->HDFS pipeline from LInkedIn. It is a mapreduce job that does distributed data loads out of Kafka.
7
stars
4
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Set Service line to Data Science & Engineering
#157
shopify-services
closed
6 years ago
1
Action required: CircleCI v1.0 is EOL, this repo is impacted
#156
ayatsynych
closed
6 years ago
1
BB
#155
Brideau
closed
6 years ago
0
No SLOs defined
#154
shopify-services
opened
6 years ago
0
[staging] Rollback on failed tasks
#153
dterror-zz
closed
6 years ago
0
[WIP] Using retries when committing files to the final destination
#152
dterror-zz
opened
6 years ago
2
Remove uploadToGcs job from Azkaban
#151
dterror-zz
closed
6 years ago
0
Extract file uploading to GCS to a separate method and adding re-tries
#150
dterror-zz
closed
6 years ago
1
Removing Watermark related jobs
#149
dterror-zz
closed
6 years ago
5
Rollbacks on failed tasks
#148
dterror-zz
closed
6 years ago
4
Remove the upload and reconciliation tasks.
#147
olessia
closed
6 years ago
0
Remove the upload and reconciliation tasks.
#146
olessia
closed
6 years ago
0
Offset patcher
#145
olessia
opened
6 years ago
1
Upload after commit.
#144
olessia
closed
6 years ago
0
Upload after commit.
#143
olessia
closed
6 years ago
0
Remove dir creation.
#142
olessia
closed
6 years ago
0
Report error to statsd.
#141
olessia
closed
6 years ago
0
test
#140
olessia
closed
6 years ago
0
Copy filesystem.
#139
olessia
closed
6 years ago
0
Copy configuration.
#138
olessia
closed
6 years ago
0
Set the prefix for gcs.
#137
olessia
closed
6 years ago
0
Upload after commit
#136
olessia
closed
6 years ago
0
Sending StatsD events for non-fatal errors
#135
dterror-zz
closed
6 years ago
5
Add line ending.
#134
olessia
closed
6 years ago
0
Topics line ending
#133
olessia
closed
6 years ago
0
Improvements to gcs upload.
#132
olessia
closed
6 years ago
0
Gcs reconciler
#131
olessia
closed
6 years ago
0
Gcs reconciler: parse properties file
#130
olessia
closed
6 years ago
0
Staging fixes.
#129
olessia
closed
6 years ago
0
Gcs reconciler staging fixs
#128
olessia
closed
6 years ago
0
Improvements to gcs upload.
#127
olessia
closed
6 years ago
0
Add hdfs <-> gcs reconciler.
#126
olessia
closed
6 years ago
0
Changing from overwriting files to swapping symlinks
#125
dterror-zz
closed
6 years ago
0
Changing from overwriting files to swapping symlinks
#124
dterror-zz
closed
6 years ago
1
Report each file that is late, and the count of records in that file.
#123
olessia
closed
6 years ago
0
Do not overwrite existing flag.
#122
olessia
closed
6 years ago
1
Do not overwrite existing flag.
#121
olessia
closed
6 years ago
0
Better gcs upload script.
#120
olessia
closed
6 years ago
0
Better upload sanity check for gcs upload
#119
olessia
closed
6 years ago
0
Better upload sanity check for gcs upload
#118
olessia
closed
6 years ago
0
Fix number of mappers for distcp.
#117
olessia
closed
6 years ago
0
Upload to gcs
#116
olessia
closed
6 years ago
0
Upload to gcs
#115
olessia
closed
6 years ago
0
Script to upload Camus dropped dirs to gcs.
#114
olessia
closed
6 years ago
2
Merge master into staging
#113
olessia
closed
6 years ago
0
Create a new Azkaban task for deduplication.
#112
olessia
closed
6 years ago
0
Deduplication task
#111
olessia
closed
6 years ago
0
Check for LAD only in the dirs written in the latest run.
#110
olessia
closed
6 years ago
1
Delay watermarking by 4 hours.
#109
olessia
closed
6 years ago
0
Delay watermarking by 4 hours; do not alert on LAD
#108
olessia
closed
6 years ago
0
Next