search

AbsaOSS / cobrix

A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
Apache License 2.0
138 stars 77 forks source link

ADLS support #636

Closed vanessakt closed 1 year ago

vanessakt commented 1 year ago

Hello,

I'm looking to load mainframe files that are stored in ADLS to process them with Cobrix.

Every time I attempt to specify an ABFSS path (Azure Data Lake Storage), and run the read, an error is generated indicating incorrect configuration settings for connecting to ADLS containers. I've conducted tests by loading files using an ABFSS path without Cobrix, and it functions correctly. This suggests that the issue might be related to the Cobrix libraries, possibly lacking support for Cloud Storage reads of this nature.

Could you please help me?

I saw there was another post related to this back in 2021, I was wondering if there was an update related to this.

Thanks,

vanessakt commented 1 year ago

Was able to make it work passing ADLS configs through sc.hadoopConfigurations() ... Not related to Cobrix.