search

siyi-wind / TIP

[ECCV 2024] TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data (an official implementation)
Apache License 2.0
19 stars 2 forks source link

question about the DVM dataset #5

Open ztt0821 opened 4 days ago

ztt0821 commented 4 days ago

Hi， I am use the create_dvm_dataset.ipynb to generate the required data. I found one problem: There is no saved file named"dvmfeatures{split}_noOH{v}_physical_jittered_50.csv", but the program read it .

FileNotFoundError Traceback (most recent call last) Cell In[83], line 18 16 check_or_save(reorder_field_lengths_tabular, join(FEATURES, f'tabular_lengths{v}_physical_reordered.pt'),) 17 for split in ['train', 'val', 'test']: ---> 18 data_tabular = pd.read_csv(join(FEATURES, f'dvmfeatures{split}_noOH{v}_physical_jittered_50.csv'), header=None) 19 reorder_data_tabular = data_tabular.iloc[:, reorder_ids] 20 check_or_save(reorder_data_tabular, join(FEATURES, f'dvmfeatures{split}_noOH{v}_physical_jittered_50_reordered.csv'), index=False, header=False)

File /research/d1/rshr/ttzhang/anaconda3/envs/tip/lib/python3.9/site-packages/pandas/util/_decorators.py:211, in deprecate_kwarg.._deprecate_kwarg..wrapper(*args, *kwargs) 209 else: 210 kwargs[new_arg_name] = new_arg_value --> 211 return func(args, **kwargs)

File /research/d1/rshr/ttzhang/anaconda3/envs/tip/lib/python3.9/site-packages/pandas/util/_decorators.py:331, in deprecate_nonkeyword_arguments..decorate..wrapper(*args, *kwargs) 325 if len(args) > num_allow_args: 326 warnings.warn( 327 msg.format(arguments=_format_argument_list(allow_args)), 328 FutureWarning, 329 stacklevel=find_stack_level(), 330 ) --> 331 return func(args, **kwargs)

File /research/d1/rshr/ttzhang/anaconda3/envs/tip/lib/python3.9/site-packages/pandas/io/parsers/readers.py:950, in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, error_bad_lines, warn_bad_lines, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options) 935 kwds_defaults = _refine_defaults_read( ... 863 else: 864 # Binary mode 865 handle = open(handle, ioargs.mode)

FileNotFoundError: [Errno 2] No such file or directory: '/research/d1/rshr/ttzhang/dataset/tabular/features2/dvm_features_train_noOH_all_views_physical_jittered_50.csv'

siyi-wind commented 2 days ago

Hi, dvm_features_train_noOH_all_views_physical_jittered_50.csv is created and saved through this code

image

You can create dvm_features_train_noOH_all_views_physical_jittered_50.csv, dvm_features_train_noOH_all_views_0.1_physical_jittered_50.csv, and dvm_features_train_noOH_all_views_0.01_physical_jittered_50.csv by changing k=['', '_0.1', '_0.01']