-
In the readme, in comparisons against each of Avro, Protobuf, JSON, and Flatbuffers, each comparison lists "Safely handle untrusted data" as a benefit of Noproto. I can't speak for Avro or FlatBuffers…
-
As the WES specification gains traction and there are more adopters, it is becoming apparent that interoperability between WES implementations is a real challenge. Initiatives like FASP have highlig…
-
**Less ambiguity in downloaded datasets**
In each downloaded dataset, repeat its DOI, time/date of extraction (as in the dialog box) and enumerate the corresponding search terms and filters.
-----
…
-
## Product outcome
Currently, internal arrangement sizes are reported only in relation to the rendered dataflow graph that creates them. Consequently, it is hard to figure out which parts o…
-
It would be great if this library could automatically create batches & save them using SequenceExample. I tried to batches myself, but I got memory issues when trying to do so. I think if it was handl…
-
The [BigQuery Storage API](https://cloud.google.com/bigquery/docs/reference/storage) provides fast access to BigQuery-managed storage directly.
Right now, reading data from BigQuery would involve e…
-
**Describe the problem you faced**
Migrated to Hudi 0.14.0 from Hudi version 0.8. While attempting to read Hudi table in parquet format from AWS S3 using Spark receiving the following error:-
…
-
Clustering/Compaction job throw follow exception, the final result returns -1 and the job's state is success.
ERROR UtilHelpers: Cluster failed
org.apache.spark.SparkException: Job aborted due to …
0xDTE updated
9 months ago
-
It seems like there is some remaining work in order to have a completely good story around schema validation. I think our first goal should be to have easy out of the box support for the functionality…
-
**Environment setup: AWS EMR serverless 6.9.0 version
Pyspark ETL job with multiple streaming queries, each streaming query writes to an iceberg table and redshift table, in microbatches, the trigger…