About Unit test

[Priyanka-43]

@kylemasc917 Hello, I wanted to ask about the unit test part. Shall we used pytest an unit test libraries to make sure it will be running or if it is already giving the results so I am done with the unit test part?

[kylemasc917]

You have to write a unit test using pytest even if the code is running correctly as a sort of sanity check to make sure it behaves as it should.

[Priyanka-43]

I have tried to write code. It is running, but still showing some error.

[kylemasc917]

I have never encountered this error before and without seeing the code that caused it I cannot tell you more beside that you can probably look up the error on google and others who have encountered have shared their solutions.

[avgagliardo]

That seems like some sort of pathing issue. Are you running this locally or in the codespace?

[Priyanka-43]

@avgagliardo I am running this code in google collab.

[avgagliardo]

Hmm... can you link me to it? I am not really familiar with the platform so they might be using a different type of container where it might not play nicely with this type of unit testing.

Usually when you run unit tests you start a worker process from the command line (by running python yourUnitTestFilename.py ) which then goes through all the automated testing for you and catches the errors.

If google collab won't allow you to start a new process or execute files directly, you might run into this sort of issue.

[Priyanka-43]

I think by mistake I link this to main branch. Can you see?

[Priyanka-43]

Or I think you can see in my repository.

[avgagliardo]

I'm not seeing it in, can you link the direct file?

[Priyanka-43]

How can i link it here, i i have written code in google collab?

[avgagliardo]

There should be some way to share the workspace or link the document. I'm not really sure, I don't use that platform so I'm not familiar with its interface.

[avgagliardo]

Check the upper right hand part of the interface for a share button. I think it might be up there somewhere.

[Priyanka-43]

https://colab.research.google.com/drive/13ztAWW_kuZlxosXTXqqefaU6fFs7800U?usp=sharing Can you see if it works?

[avgagliardo]

Its not public, but I just requested access.

[Priyanka-43]

I accept.

[avgagliardo]

Okay I think I see part of the problem. The whole project lives inside of a single google collab notebook (which is like a jupyter notebook), which is good for trying things out but certain things (especially unit tests) should be broken out into separate files. This is the short version.

The slightly more verbose version is that, since your code lives in a notebook instead of a first-order python file, it is causing trouble when it comes to the platform starting the testing process, also it is going to very difficult to import it into your unit-test.py file.

It would probably save you the most amount of time to if you were to copy and paste all of the relevant code into its own .py file(s) depending on how you want the module to work. Once all of that code lives in python files (the unit tests too), then you should be able to execute the code directly and run the unit tests without these types of issues.

[Priyanka-43]

Ok, but if you will scroll down, first i tried to run in separate file. then it did not work. Then i try to do in single file, then also it was causing problem.

[avgagliardo]

I think you are talking about the code cell right? Those aren't separate files, those are just small blocks of code that reside within the same ipython notebook.

Maybe I'm missing other files though so I'll go double check.

[avgagliardo]

The only other files that I see are .csv and .json data stores so I think you might be mistaking files with how notebooks work. This is one of the more deceptive parts of using python and jupyter.

[Priyanka-43]

Yeah, I was talking about cell.

[avgagliardo]

Okay, that's where the issue is. Your project lives in a single ipython/jupyter notebook (they have the file extension .ipynb), these issues can be fixed if you split it up into separate python files (.py file extensions).

[Priyanka-43]

Like, I am new to all these platforms. I am still unable to resolve this issue.

[kylemasc917]

You need to rewrite your single Jupyter Notebook file as separate python files. In GitHub this can be done in a codespace by typing in the top search box >New clicking new file and selecting a .py file and rewriting your code in there.

[ojha-aditya]

I guess that can be done in collab also, there was some discussion in last class about saving Jupyter notebooks as python files.

[ojha-aditya]

Another way is to use the terminal to directly convert the notebook using nbconvert command. I am not sure about the collab interface though, might be slightly different than codespaces.

[avgagliardo]

You can convert the files, but they still will not be well-formed python executables. conversion is a risky proposition, it rarely works out exactly how you would hope.

I would just use what you have and create new .py files by copying out the relevant code. From there you can manually run and test them in the codespace.

Tl;Dr: breaking your notebook into individual python files will fix it

[avgagliardo]

Like, I am new to all these platforms. I am still unable to resolve this issue.

Check out issue #57 , it might help you distinguish between the different types of files and platforms.

[ojha-aditya]

I see, didn't know about that @avgagliardo. What issues come up though, while converting? Is it largely formatting related or more serious than that?

[avgagliardo]

I see, didn't know about that @avgagliardo. What issues come up though, while converting? Is it largely formatting related or more serious than that?

Formatting with respect to indentation and things like that should be trivial and handled without much issue during the conversion process.

What the real issue is that there is not a great way for the convertor to understand how a notebook, which is typically written (and expected to run) as a sequential set of code cells, should be reconciled into a module, an object that does not necessarily have have any sort of linearity prescribed for its architecture or usage.

That means that if you do certain things across multiple cells ( ie, multiple imports, define multiple functions that share names, declare objects repeatedly, set or update their values) all of that will be automatically converted (or at least attempt to convert) it into a single .py file which ultimately will not be well-formed. As a result, it will likely not execute and it will be much harder to understand what is going on compared to the .ipynb version or if you had tried to implement the functionality as a .py, first and then test it in a .ipynb notebook.

Notebooks are SUPER useful, but the trick is not doing your full module development inside of them. Once you reach the point of producing functional code, its best to extract(or even abstract if you are making a more complex system) it and place it into a sensible spot in whatever module you are developing. This is actually one of the things that TDD (Test-Driven Development, see #11 ) can help with, because you have a tangible point in the dev cycle where you know the code works-- which is a good signal that it might be time to formalize it by putting it into a module file instead of keeping it in a notebook.

I tried to keep it pretty general, since there are always exceptions, but if you have any questions feel free to ask -- but like always I definitely don't claim to be an expert or anything like that, I've just been using python for a while.

[dnxjay]

I have tried to write code. It is running, but still showing some error.

were you able to fix this error? if so how did you go about doing so?

[ojha-aditya]

I see, didn't know about that @avgagliardo. What issues come up though, while converting? Is it largely formatting related or more serious than that?

Formatting with respect to indentation and things like that should be trivial and handled without much issue during the conversion process.

What the real issue is that there is not a great way for the convertor to understand how a notebook, which is typically written (and expected to run) as a sequential set of code cells, should be reconciled into a module, an object that does not necessarily have have any sort of linearity prescribed for its architecture or usage.

That means that if you do certain things across multiple cells ( ie, multiple imports, define multiple functions that share names, declare objects repeatedly, set or update their values) all of that will be automatically converted (or at least attempt to convert) it into a single .py file which ultimately will not be well-formed. As a result, it will likely not execute and it will be much harder to understand what is going on compared to the .ipynb version or if you had tried to implement the functionality as a .py, first and then test it in a .ipynb notebook.

Notebooks are SUPER useful, but the trick is not doing your full module development inside of them. Once you reach the point of producing functional code, its best to extract(or even abstract if you are making a more complex system) it and place it into a sensible spot in whatever module you are developing. This is actually one of the things that TDD (Test-Driven Development, see #11 ) can help with, because you have a tangible point in the dev cycle where you know the code works-- which is a good signal that it might be time to formalize it by putting it into a module file instead of keeping it in a notebook.

I tried to keep it pretty general, since there are always exceptions, but if you have any questions feel free to ask -- but like always I definitely don't claim to be an expert or anything like that, I've just been using python for a while.

Oh I understand now, it makes sense. Thank you for the explanation.