tsv_split_merge

[Amnesiae]

Can you provide the merged flickr tsv dataset? the file I merged using the tsv_split_merge.py does not work

[Snak0067]

yes,I also tried to merge the downloaded dataset, but this error occurred. What could be the problem? Traceback (most recent call last): File "/mnt/workspace/workgroup/chenlei/paper/GLIGEN/tsv_split_merge.py", line 325, in merge(args.merge_in_folder, args.merge_out_folder) File "/mnt/workspace/workgroup/chenlei/paper/GLIGEN/tsv_split_merge.py", line 289, in merge for idx in range(len(reader)): File "/mnt/workspace/workgroup/chenlei/paper/GLIGEN/tsv_split_merge.py", line 137, in len return self.num_rows() File "/mnt/workspace/workgroup/chenlei/paper/GLIGEN/tsv_split_merge.py", line 102, in num_rows self._ensure_lineidx_loaded() File "/mnt/workspace/workgroup/chenlei/paper/GLIGEN/tsv_split_merge.py", line 145, in _ensure_lineidx_loaded self._lineidx = [int(line) for line in lines] File "/mnt/workspace/workgroup/chenlei/paper/GLIGEN/tsv_split_merge.py", line 145, in self._lineidx = [int(line) for line in lines] ValueError: invalid literal for int() with base 10: '0\t{"data_id": 0, "image": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQgJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjI

[Hzzone]

Modify these lines:

  # lineidx_files = files[0:N//2]
  # tsv_files = files[N//2:N]
  lineidx_files = sorted([f for f in files if f.endswith('lineidx')])
  tsv_files = sorted([f for f in files if f.endswith('tsv')])