sassoftware / python-dlpy

The SAS Deep Learning Python (DLPy) package provides the high-level Python APIs to deep learning methods in SAS Visual Data Mining and Machine Learning. It allows users to build deep learning models using friendly Keras-like APIs.
Apache License 2.0
224 stars 131 forks source link

load audio files server and local path issues #376

Open TomStockAnalytics opened 2 years ago

TomStockAnalytics commented 2 years ago

modelPath = '/srv/nfs/kubedata/cas-landingzone/sbxtot/sounds'

s.addcaslib(activeonadd=True,datasource={'srctype':'path'},name='mycaslib',path=modelPath,subdirectories=True) s.setsessopt(caslib='mycaslib')

local_data_test = '/srv/nfs/kubedata/cas-landingzone/sbxtot/sounds/test' server_data_test = '/srv/nfs/kubedata/cas-landingzone/sbxtot/sounds/test' local_data_train = '/srv/nfs/kubedata/cas-landingzone/sbxtot/sounds/train' server_data_train = '/srv/nfs/kubedata/cas-landingzone/sbxtot/sounds/train' local_data_validate = '/srv/nfs/kubedata/compute-landingzone/sbxtot/sounds/validate' server_data_validate = 'srv/nfs/kubedata/compute-landingzone/sbxtot/sounds/validate'

audio_table_train = AudioTable.load_audio_files(s, casout={'name':'audio_files_train','replace':'True'},
local_audio_path=local_data_train, server_audio_path=server_playground_data)

I can't understand why this procedure can't process those files and constantly throw that error:

Cannot convert file /srv/nfs/kubedata/cas-landingzone/sbxtot/sounds/train/abnormal/00000110.wav Number of files processed: 0 File conversions are finished. ERROR: A PATH type caslib is required when using filetype DOCUMENT. ERROR: The action stopped due to errors. ERROR: A PATH type caslib is required when using filetype DOCUMENT. ERROR: The action stopped due to errors.

While in the meantime, the following procedure works and create my table but it includes all the subdirectories of sounds (test, train, validate) content in one file

rt2 = s.retrieve('table.loadtable', _messagelevel='error', casout= {'name':'audio_files_train','replace':'True'}, caslib='mycaslib', path='', importOptions=dict(fileType='AUDIO', contents=True, recurse=True))

image

Can you let me know how to solve this issue ? Thanks, Tom.