-
so i want to know why there will be some parquet files which the file name start with dot(.) when I write data to hudi? And how to filter these files when I read hudi by spark? Thank you very much!
-
**Type**: Feature
**Is your feature request related to a problem? Please describe.**
I have found that several people i work with have assumed that S3Resource.getUri() would return the s3 url for …
-
This is available already at the file level:
https://github.com/apache/parquet-cpp/blob/master/src/parquet/file/metadata.h#L177
but not at the ColumnChunk level
**Reporter**: [Wes McKinney](https:/…
-
### Backend
VL (Velox)
### Bug description
I am using Gluten(tag v1.1.1) + Velox + folly + spark3.4.2 + yarn，building with centos8 aarch64, running on aarch64 , while yarn container running with …
-
i have trained a model using python3.7 and tf 2.7 , save the model by 'saved model' format , like this:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['…
-
### Describe the problem you faced
When storing a data structure with the following layout into a copy-on-write table:
```
root
|-- personDetails: struct (nullable = true)
| |-- id: intege…
-
**Task Description:**
Training a simple classifier using keras + horovod spark and getting below error
**Error:**
```
[3]:Error in sys.excepthook:
[3]:
[3]:Original exception was:
[3]:#
…
-
### Description
Client should support JSON Data type (https://clickhouse.com/docs/en/integrations/data-formats/json/overview).
**Considerations:**
- Client should provide convenient API to get a…
-
I'm using parquet on Windows 10 and I have two different parquet files for testing, one is snappy-compressed, one is not compressed.
Simple test code for reading:
with open(filename,'r') as …
-
## Problem
### How much memory does a spark-dependencies job take while handling about 12Gb data index?
I am totally new to the spark project and I have tried serval times to run a spark-depend…