linkedin spark-tfrecord issues

linkedin / spark-tfrecord

Read and write Tensorflow TFRecord data from Apache Spark.

BSD 2-Clause "Simplified" License

290 stars 57 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

spark-tfrecord writes not working for spark 3.5.0

#72 lzuwei closed 2 months ago
4
spark-3.5 branch

#71 IgorBerman closed 7 months ago
1
Does not support Nested ArrayTypes

#70 apurva-sharma closed 9 months ago
2
NullPointerException at org.apache.hadoop.conf.Configuration

#69 Yingminzhou opened 11 months ago
1
java.lang.ClassNotFoundException: Failed to find data source: tfrecord. Please find packages at http://spark.apache.org/third-party-projects.html

#68 Ethanhack opened 1 year ago
3
Decoding a BytesList

#67 albertoandreottiATgmail closed 1 year ago
3
Support for reading ByteArray's from TFRecord files

#66 asketagarwal closed 1 year ago
1
Adding support for reading ByteArray's from TFRecord files

#65 asketagarwal closed 1 year ago
3
make spark 3.4.0 compatible

#64 ssiegel95 closed 1 year ago
9
version bump

#61 junshi15 closed 1 year ago
0
Support Byte Array write output format

#60 yozenliu closed 1 year ago
2
Serializing ByteArray without TFExample feature wrapping

#59 yozenliu closed 1 year ago
3
Spark 3.3 support?

#58 InvisibleMan1306 opened 1 year ago
4
The extension name of the saved file is '.tfrecord.snappy'. Failed to load the file using tensorflow==2.4

#57 gdtydm closed 2 years ago
1
ArrayType(ArrayType(DoubleType,true),true) Not Supported When Writing Dataframe to TFRecords

#56 xemcerk opened 2 years ago
2
How to convert spark.ml.linalg.VectorUDT datatype to tfrecord?

#55 mullerhai closed 2 years ago
2
spark 3.1.1 Caused by: java.lang.ClassNotFoundException: tfrecords.DefaultSource

#54 mullerhai closed 2 years ago
5
Do we support spark dataframe straightly convert to tensorflow-java Tensor like TFloat32 TFloat64 or Operand[T <: TNumber]

#53 mullerhai opened 2 years ago
1
Support Spark 3.2 + Scala 2.13

#52 junshi15 closed 2 years ago
0
[Feature Request] Add option to batch data when using SequenceExample

#51 utkarshgupta137 opened 2 years ago
5
Spark 3.2

#50 ingolfured closed 2 years ago
2
Support for Spark 3.2.0+ with scala 2.13

#49 sheilahu opened 2 years ago
2
spark 读取 tfrecord文件，解析tf.train.Feature(bytes_list=tf.train.BytesList(np.array(feature, dtype=np.float32).tobytes())) 文件字节码异常

#48 PeterGss opened 2 years ago
1
TFRecords File is too big! 10X the size of parquet

#47 kart2k15 opened 2 years ago
2
tfrecord write results in no data but no error

#46 dennisobrien opened 2 years ago
2
Read tfrecord by tensorflow

#45 yiwei00000 closed 2 years ago
0
Error: java.lang.ClassCastException: com.linkedin.spark.shaded.org.tensorflow.example.FeatureList cannot be cast to com.linkedin.spark.shaded.org.tensorflow.example.Feature

#44 nitinware opened 2 years ago
3
write compression none invalid

#43 wangxiao111 opened 2 years ago
1
How to save tfrecord data in SequenceExample format？

#42 ZhouM1118 closed 2 years ago
4
How to save TFRecord from RDD[Example]?

#41 wuxianxingkong closed 2 years ago
2
[ERROR] Some problems were encountered while processing the POMs

#40 DachuanZhao closed 2 years ago
3
Dataframe can not convert to tfrecord

#39 Brian1203-zz opened 2 years ago
3
Add syntax highlighting in readme python block

#38 jeffistyping closed 2 years ago
0
fix OutputWriter break in spark 3.2

#37 tangyl closed 2 years ago
2
Writing TFRecords breaks in Spark 3.2.0

#36 M-Anwar closed 2 years ago
5
encountered a FileAlreadyExistsException when trying to save array of arrays of arrays (3d array)

#35 GraceHLiu opened 3 years ago
1
Consume TensorFlowInferSchema from Java Spark 2.4.0

#34 yeshbash opened 3 years ago
2
Enable compatibility with OCI HDFS Connector

#33 magruener closed 3 years ago
4
Error while reading TFRecords - buildReader is not supported for TFRECORD

#32 GaganSD closed 3 years ago
5
fix issue #27

#31 junshi15 closed 3 years ago
0
fix issue#27

#30 junshi15 closed 3 years ago
0
version bump

#29 junshi15 closed 3 years ago
0
fix for issue-27, filtering files by schema in addition to file size

#28 junshi15 closed 3 years ago
0
schema inference failed when read compressed saved data

#27 lxx719 closed 3 years ago
8
Fixed deserializer for case when sequential rows have different features

#26 schCRABicus closed 3 years ago
6
Protocol message tag had invalid wire type.

#25 ak2911 closed 2 years ago
5
how to save Rdd to TFRecord in a distribution environment?

#24 ICDI0906 closed 2 years ago
3
I not understand how to run it , use java -cp jar

#23 TIXhjq closed 2 years ago
3
java.lang.RuntimeException: unsupported type ...

#22 seiyagoo opened 3 years ago
6
HELP: how to support write option maxRecordsPerFile?

#21 duanbing closed 3 years ago
1