After syncing with @katrinakalantar I realize I misinterpreted this code and introduced a logical error. I also realized all this code could be replaced with panda's built-in methods. I think this makes things much simpler and more performant. I tested locally on a file @katrinakalantar provided and it worked. She mentioned some tsv parsing weirdness with these files so we can keep an eye out when testing the workflow as a whole and address any issues as they arise.
@katrinakalantar thanks actually I will add a comment in the code as well. If there is weirdness we can use pandas' regex separator so the code won't change too much.
After syncing with @katrinakalantar I realize I misinterpreted this code and introduced a logical error. I also realized all this code could be replaced with panda's built-in methods. I think this makes things much simpler and more performant. I tested locally on a file @katrinakalantar provided and it worked. She mentioned some tsv parsing weirdness with these files so we can keep an eye out when testing the workflow as a whole and address any issues as they arise.