-
Decide on a code formatting standard:
Spacing - Spaces vs Tabs
Function / Class / Namespace Naming
Any other suggestions for standardisation?
-
- [ ] accessanalyzer →
- [ ] acm →
- [ ] acm-pca → #13327
- [ ] alexaforbusiness →
- [ ] amplify →
- [ ] amplifybackend →
- [ ] apigateway →
- [ ] apigatewaymanagementapi →
- [ ] apigatewayv2 →
- [ ] …
-
- [x] 출발지 S3의 최신 변경 사항을 감지후(DB 기록 사항과 비교) 부분 작성
- [x] 목적지 S3에 COPY 부분 작성
- [x] 목적지의 신규 데이터 경로 DB에 insert 부분 작성
- [x] Spark가 데이터를 받아서 train.json, test.json을 수행하는 Application 실행 부분 작성
- #7 …
-
I creat a webdataset below, where ResampledShards is defiend to repeat tar files to make sure every gpu and every worker could load different tar file. But i found that dataset will not load the compl…
-
It is data genesis!
### Which problem is this feature request solving? ###
- Where and from whom data came from.
- Currently ODPS does not offers any attributes or mechanism to define it.
-…
-
Code-driven data pipeline to take data from Google Drive (mix of Google Sheets, Excel files, and folders containing those), and land it in BigQuery dataset `tag-dssg-2023-lbc-all-teams.data_raw` with …
-
Following the development of https://github.com/WorldBank-Transport/ram-backend/issues/229 I noticed that the [default profile file](https://github.com/WorldBank-Transport/ram-backend/blob/c2b441dafb2…
-
We had some discussion in `#datapipeline` in IRC about how explicitly setting a number of shards to write is necessary for unbounded sources, and setting an explicit number of shards incurs a GroupByK…
-
Jenkins で実行しているジョブの一部を AWS Batch に移行できないか検討する。
すべてを移せるとは思っていないが、Jenkins じゃないとダメなもの以外は Jenkins 以外に移行したい。
そうすれば Jenkins も管理しやすくなる。
ちなみにその他のサービスとしては
* AWS CodeBuild
* AWS StepFunction
* AWS Dat…
-
Currently the generator doesn't appear to query for the exist-db version during the build process. In my particular case I'm using exist-db 6.0.1, and was using the latest exist-db yeoman generator. T…