search

haochengxia / vscode-pydata-viewer

Display PyData files (.npz .npy .pkl .pth) in VSCode.
https://marketplace.visualstudio.com/items?itemName=Percy.vscode-pydata-viewer
MIT License
14 stars 8 forks source link

content missed while loading a .pkl file #26

Open bg-szy opened 2 months ago

bg-szy commented 2 months ago

While loading a .pkl file, there were some data located at the tail part of the file turned invisible though i could print it out no matter the file size is tens of kb or hundreds of kb

bg-szy commented 2 months ago

If you could answer, I would be very grateful to you

haochengxia commented 2 months ago

Thanks for your findings and for kindly reporting it! Can you provide screenshots, instructions, or files to help reproduce the bug?

bg-szy commented 2 months ago

Thanks for your findings and for kindly reporting it! Can you provide screenshots, instructions, or files to help reproduce the bug?

Sorry i'm not allowed to provide my data here. The file contains about 200 frames of data, each of them contains the same K-V pairs. But it just can't show all of them especially at the end part of the file like after frame 80 or more

haochengxia commented 2 months ago

Thanks for your findings and for kindly reporting it! Can you provide screenshots, instructions, or files to help reproduce the bug?

Sorry i'm not allowed to provide my data here. The file contains about 200 frames of data, each of them contains the same K-V pairs. But it just can't show all of them especially at the end part of the file like after frame 80 or more

Thanks for further information! It is greatly helpful for me to figure out the issue. I will fix it as soon as possible.

haochengxia commented 2 months ago

I tried simulating the described scenario using the provided code but was unable to reproduce the issue. I kindly request further instructions or comments on this problem. Thanks.

import pickle

num_frames = 200

keys = ["key1", "key2", "key3", "key4"]
dummy_values = range(1, 5)

output_file_path = "data_output.pkl"

data = []

for frame_number in range(num_frames):
    frame = {f"{key}": f"value{value}_{frame_number+1}" for key, value in zip(keys, dummy_values)}
    data.append(frame)

with open(output_file_path, 'wb') as f:
    pickle.dump(data, f)

print(f"Write completed.")
bg-szy commented 2 months ago

I tried simulating the described scenario using the provided code but was unable to reproduce the issue. I kindly request further instructions or comments on this problem. Thanks.

It seems I didn't describe the problem accurately. This may just be a display issue, I can't fully display the contents of my file when using this plugin in vscode

haochengxia commented 2 months ago

I understand. Is the content truncated after a certain size for display? If so, I will check the display limitation of the webview in VSCode.

bg-szy commented 2 months ago

I understand. Is the content truncated after a certain size for display? If so, I will check the display limitation of the webview in VSCode.

I'm not sure about this, but I have had this problem in files of tens of kb and hundreds of kb, and they all appear at the back of the file. It's not that the data behind is not displayed, but that some are displayed and some are not show