combination multiple small files together faced problem: OSError: [Errno 27] File too large

[HuixuanGan]

Code snippet

def merge_mdf_files(file_paths):

Initialize the first MDF object

mdf_list = []
# merged_mdf = asammdf.MDF(file_paths[0])

print(file_paths)
# Load subsequent MDF files and append them to the first
for file_path in file_paths:

    if os.path.exists(file_path): 

        mdf = asammdf.MDF(file_path)

        print(f"File found: {file_path}")  
        # This should append the contents of mdf to merged_mdf

    mdf_list.append(mdf)

merged_mdf = asammdf.MDF.concatenate(mdf_list)  

Description

The fastest way to debug is to have the original file. For data protection you can use the static method scramble to scramble all text blocks, and send the scrambled file by e-mail.

I wanted to combine about 700 MF4 files together which have about 320MB, however, I faced such error, could you tell me how to resolve this?

[danielhrisca]

Share the complete error traceback mesage

[HuixuanGan]

Traceback (most recent call last): File "/mnt/d/Huixuan_gan/Analyze_Optimal/separate_thread_ver/test_lastday/run.py", line 733, in last_report("2024-06-19T00:00", "2024-06-20T00:00") File "/mnt/d/Huixuan_gan/Analyze_Optimal/separate_thread_ver/test_lastday/run.py", line 713, in last_report worker1.merge_and_decode_list(local_mf4, uploadname) File "/mnt/d/Huixuan_gan/Analyze_Optimal/separate_thread_ver/test_lastday/run.py", line 75, in merge_and_decode_list merged_mdf = merge_mdf_files(mf4_paths) File "/mnt/d/Huixuan_gan/Analyze_Optimal/separate_thread_ver/test_lastday/test_assdecode.py", line 52, in merge_mdf_files merged_mdf = asammdf.MDF.concatenate(mdf_list) File "/mnt/d/Huixuan_Gan/Analyze_Optimal/separate_thread_ver/venv/lib/python3.10/site-packages/asammdf/mdf.py", line 2583, in concatenate merged.extend(cg_nr, signals) File "/mnt/d/Huixuan_Gan/Analyze_Optimal/separate_thread_ver/venv/lib/python3.10/site-packages/asammdf/blocks/mdf_v4.py", line 5910, in extend stream.seek(0, 2) File "/usr/lib/python3.10/tempfile.py", line 622, in func_wrapper return func(*args, **kwargs) OSError: [Errno 27] File too large

[danielhrisca]

https://stackoverflow.com/questions/10956175/file-too-large-python

Are you using a file system that does not support large files?

[HuixuanGan]

Found problem in my system setting and fixed, thanks!