nigleweber
commented
2 years ago Just having a quick read through your comment + code tonight. I'll read through your comment more tomorrow.
I'll answer the questions quickly:
- Nothing wrong with the function. It's got some inner logic that'll be great to convert into a helper function. At the end we'll do a review of your code and see what we can add to your helper.py file + a more in-depth explanation of why this is important.
- For the weekend I'll not have nearly the same time to review + answer. That being said I'll always do my best to respond if there's anything urgent.
- Two things on the deadline for this: first I gave ya two weeks and said "You could probably do it in one week" so you're already ahead from the looks of things. Second don't worry about the deadlines too much, we'll discuss this briefly but I'll also explain below.
Don't sweat the deadlines I give you much Enzo. If I can see you're clearly making the effort and/or spending 4 or more hours on our projects I'm not gonna give you a hard time. 😄 Doing something the right way is usually better than cutting corners to meet the deadline. To elaborate a bit on why I gave and will continue giving you deadlines is that I've (tried) training others who didn't really care all that much. Having a deadline makes it easy to "prove" they aren't interested in they aren't putting in the effort. So far you seem to be doing great in terms of effort. 🏅 The other reason for deadlines is that it's something you're going to face in the industry all the time. For now I'll handle deadlines the way I mentioned, if later on you want practice dealing with missed deadlines we can do that too.
For the next meeting I'll send you a plan of how I envision our interactions working. I'll add more details but the short version will be:
- 30 minute meetings weekly during a project to discuss any issues + progress
- A 60 minute meeting to review + assign any remaining work
- Regular 30 minute meeting to discuss next project or going back over the current project
Rinse and repeat 1-3. Obviously there'll be variations from time to time if we're working on your resume or holidays etc.
FIX: 1) Update return for every function. 2) Parsing done normally it's working perfectly. 3) Variable CAPS still not fixed, after your response (check last closed PR) I'll change things up or I'll let them stay like this 4) Removed the pycache stuff
New features / Improvement: 1) With the return fix on every function, I also removed every exit(1) on the functions, especially in get_path(), easier to return for any problem and have only one exit(1) in main when the function return nothing. I've done this for every function the exit() will always be in main for now on.
2)parse_json() brings new features, tiny line to separate each parsing/ request, giving information about what string is processed, statistics about number of reports of string flagged as malicious and other things + more detailed report when malicious is actually flagged with vendor name + result of their report (type of malicious activity for example). parse_json() also managed multiple things, like if the req_api() didn't give us anything (in case user search a random string by error like aafafhofhsaoi) so it return before using resource to parse for nothing. Last thing I put a stopped for user who wants to search for certains tags so it's not a weird string in itself, actually it's pretty clear like malware botnet etc (VT call them Tag comment) but the output I got when searching them is not complicated to parse or anything just there is nothing that's interesting there, like it's just a json output of what's happening on their webpage. Basically, the API is sending me ALL the comments that you could see on a webpage with folks who put botnet or a certain word in their comment. So, I just thought most of the user will clearly search for IP address, domain, URL, hashes BUT in case they search a tag comment so something that will have a result in the API (so not a random string like fsnbjisb), I'm telling them sorry, but this feature is not implemented in this script and also go to this URL and search your word again. Okay so I told you most of the things I did in this function, sorry it's not very chronological compared to what you'll see on the script but still I wanted to explain myself.
3) I just thought about this right now but if you see weird, formatted print line well it's how the format package put it when I format my code. Again, not a thing I do it's just that it's not really good or bad but if each time I format my code and I have to like hit return even though I'll totally know that during my next formatting process the thing will roll back it's just easier for me to let it like this. (I'm just trying to anticipate your questions like what's this weird print ???? lol)
4) I also thought about another thing, When I wrote the code especially for the file argument part, I didn't think about the fact that okay we might run into a very specific situation. Well let's say the user didn't read our recommendation on the readme and give a file that okay have string on it BUT there's a blank line or multiple ones between each string the file has. Well before I've added the feature, (I'll tell you about a little bit after) before this, well first line (if there is a string but if it's blank it's even worse), it's like first request totally okay works like a charm, second one since the program was just checking next line, it was causing a bad request to be sent like something empty or \s or whatever but it just crashed the program giving us a HTTP 400 from VT and also not letting us send the other string (if there was any because we could also run in a situation where the first line is just blank, so it crashed within the first request). Anyway, after having thought about this, I put a small piece of code in the main section to remove all the blank line in the file of the person. I found a cool way of doings this, I hope you'll like it. It's not that hard but still I'm pretty happy to have fixed this very specific bug. On a side note, I just have to tell you that technically the piece of code I put is totally fine it's fixing the bug. BUT two things. First one well there's smarter way to handle the problem, the solution you'll find on this version won't be the best. Why? Well, the entire content has to be read in memory first, if the file has like hundreds of thousands of strings, it might take longer to execute the script. But still I tried with the version you'll see on the commit part with like I said literally an ENORMOUS amount of data and it seems to be fairly quick, for sure it's slower since there is more data to be processed but on my human perception, there wasn't any difference. So, after this PR I modified this piece with another version that is way better at handling/ scaling the file data BUT the linter is like NOPE your instruction is too long (it's not that long technically but still...) IMO the linter is causing so much problem for so little things that I'll totally make the next PR with the new versions of this piece of code and also maybe who knows the variable with normal letters lol. (Maybe I should still keep the linter for sure it's helpful but maybe trying to put exclusions? I don't know....)
5) Normally I should also let you know that fore sure there are things that are more important like waiting your next updates on what fix should I do etc, but I was just thinking about a little upgrade in the output style of the program, maybe if we have time? Like for example, on the get_path function when the user is hitting tab it might be cool to let him see the directory he's in for example or autocomplete when's typing a path. I never done this tbh but it might be fun and user friendly. The second thing I was thinking about is that for sure I did a work that's pretty clean compared to the thing I got before parsing so in the parse_json() I've made a tiny user friendly output like the lines stuff (like this: -------) to make a separation between each request, also noting what request is processed, using as same as the msfconsole the [*] for news, the [-] for bad news and the [+] for update / good news is clearly helping a lot in the readability of the output I'm providing. Still, I just thought that maybe some colors to those texts, could be even better (and I keep all the things I said before its's just adding another layer for better readability and nice output).
Questions: 1) First thing first, you talked about getting a look about get_path() what was the problem should we fix now I mean during our next round of you pushing update and me fixing or should we wait? 2) I'm really thankful for the help you can't even imagine but I still want to tell you that I respect the concept of weekend and people having break so should we have a rule of like contacting and giving updates / fixes to each other only during weekday and we talk again on Monday for example? Or another option will be one contact during weekend when you list a LOT of things to do so it's like getting me occupied for the weekend and we get back in contact on Monday to tell you how it went for all the tasks you assigned for the weekend? I really don't know; every solution is totally okay with me. Having your help for so much different things is already incredible for me and I can't thank you enough for your help, so it's really your call on this I don't want to steal your time especially during time that could be off for you like weekends...
3) Well, the questions might be dumb, but I don't really feel stressed about the project in itself pretty fun actually but still I told you about the deadline, like in my head the project should be done before our next call. But we're getting closer and closer to our next call so do you think we we'll have time to finish it before this deadline? I mean if not I'm sorry for not respecting my deadline that's for sure, but I just wanted your opinion on this (no pression like I'm not asking this to have more things or rushing you out quite the opposite) just to know your opinion on how much has been done and how much is going to be left before having this deadline and is it realistically achievable? (Normally yes. I hope?)
Thanks to spend time on me again, I'll never be enough grateful. Hope it wasn't too long to read, Have a great weekend!