Open hanimayahi opened 8 years ago
Nikey646
commented
8 years ago Why are we shoving such per-specific features into the general use API? It seems rather silly, and would in the long run be considered bloat for everyone except you, no?
And would it not be a security risk to put Management Board related API Endpoints onto the General use API, especially when afaik, it currently has no sort of filtering for user level/development key...
Just curious™
robotman321
commented
8 years ago They area good thoughts, but what this thread doesn't show is that we will be adding dev key levels, depending on the level will give them access to certain content and functions :D
Brad Riemann Systems Engineer FTW Entertainment LLC
From: Nikey646mailto:notifications@github.com Sent: 9/12/2015 1:47 PM To: FTW-Entertainment-LLC/AnimeFTW.tv-Sitemailto:AnimeFTW.tv-Site@noreply.github.com Subject: Re: [AnimeFTW.tv-Site] Bot-san API requirements (#77)
Why are we shoving such per-specific features into the general use API? It seems rather silly, and would in the long run be considered bloat for everyone except you, no?
And would it not be a security risk to put Management Board related API Endpoints onto the General use API, especially when afaik, it currently has no sort of filtering for user level/development key...
Just curious™
— Reply to this email directly or view it on GitHubhttps://github.com/FTW-Entertainment-LLC/AnimeFTW.tv-Site/issues/77#issuecomment-139809247.
hanimayahi
commented
8 years ago @Nikey646 The uploads board actions won't be a bloat in the long run, that can probably be useful for something else in the future.
Do you have any recommendation where the bot-san specific information should be? I thought the API would be good enough because it is essentially information from aftw, which the API is for.. I could put it in another script, but the space it would occupy would be the same and it would not make a difference in the database what so ever.
Nikey646
commented
8 years ago I really cannot see any other sort of usefulness for Management API End points other than your bot, and if brad does decide to make a master for the encoding servers, that bot. Especially since brad has already stated he does not want the Management Board to be replicated to an External Program to me previously. If i recall correctly, this was when i was only a normal staff member, and may not be true anymore
As for the bot specific information? Well, everything in your first point should be stored internally on the bot, these are all details that could easily be stored in an SQLite database, or json file. As for the second point, the prefix and episodes could be stored, yet again, with the other information locally. The final and third point could use the existing infrastructure of the Management Board, via logging in, getting the php session, passing that along to a request to the manage ajax script, with the correct $_GET and $_POST details it should easily replicate the update functionality, without being too much of a security problem, since it'll all needs to go over https anyways, and already does when we use the board.
Assuming these 3 points were the way that was chosen, the only 'addition' to the API that would need to be made (and could easily be moved elsewhere, no longer requiring any real information from the database), is outputting the information for the bot to consume, including updates. This is of course, is assuming the bot is not public facing. If the bot could be public facing, it would be a matter of just sending it new information, rather then trying to make it request information. However If i recall correctly, Brad has already stated that he does not wish for the Encoding servers to be public facing if possible
Just some ideas and thoughts, in reality if you guys choose to add these functions to the API, does not really effect or bother me, I was just curious as to why features which from my perspective would be incredibly limited use (As mentioned in my first paragraph) were going to be put into the API <3
hanimayahi
commented
8 years ago The bot is essentially a part of aftw, I wouldn't say it's a completely external separate program. If you can't see any other usefulness for management API, I still don't see why it would be a problem to make the option available for the future.
I also don't like the option of storing everything in a separate database for the bot, because I want it to easily be possible to communicate with the uploads board and the bot-san interface we'll add in management. Having the interface on the management board connect to a different database just to update information, or adding a new airing series would just be a hassle imo. Same thing for the usage of the existing infrastructure, that's way too much of a hassle, that would make more code in the node application rather than just adding a tiny bit of code for the API. For the second point, again, I don't want the whole application be separate, storing it locally won't make management for the bot easy.
Updating and adding new information might be easier by using the managements ajax scripts, but I would still need a way to read the information. I want it to be possible to read the latest episode uploaded, because what happens if someone updates the entry on the uploads board to fix a error, or to skip a episode (Perhaps the fansub won't sub episode x because it's a recap, it happens). Plus it would make it a easy way to add extra management options for the bot.
Nikey646
commented
8 years ago The bot is essentially a part of aftw, I wouldn't say it's a completely external separate program.
Is the bot not going to be running as an external separate program that just so happens to consume data from AFTW, and run encodes for AFTW based on that data? Personally i call that an external separate program lol...
If you can't see any other usefulness for management API, I still don't see why it would be a problem to make the option available for the future.
With the second part of my first paragraph, i pointed out how Brad has previously told to me not to replicate the Management board in an external program, adding the API end points to do that would make it easier to ignore Brads request (While it's more than easy enough to ignore that now, that's not the point :P). Either way, this one is entirely up to Brad, so i'm not going to touch on this portion of the discussion any further.
I also don't like the option of storing everything in a separate database for the bot, because I want it to easily be possible to communicate with the uploads board and the bot-san interface we'll add in management. Having the interface on the management board connect to a different database just to update information, or adding a new airing series would just be a hassle imo.
I stated it would be stored in a SQLite Database or a Json file, these would be for the Bot only, and the Bot would retrieve new information from AFTW, then update the storage method chosen, why would the AFTW site be updating the Bots data? Why not just output new information on the Entries for the bot at a single Endpoint, that the bot consumes the new data from and stores it?
Same thing for the usage of the existing infrastructure, that's way too much of a hassle, that would make more code in the node application rather than just adding a tiny bit of code for the API.
Sorry to say, but it's not hard to make 2 Web Requests, attaching some Cookies and Post Data onto the second one from the first...In fact, the code would likely be about the same amount either way you do it...Especially considering from what i've seen there would be no easy way to just call a function in one of the Management board's classes at an API Endpoint...
storing it locally won't make management for the bot easy.
I think you are underestimating how much easier it is to change how you consume data, rather than everything because you want to change the way data is handled... Not sure how to explain this correctly, sorry.
I want it to be possible to read the latest episode uploaded,
I'm not entirely sure what you are planning that requires the current episode number from the management board, but it really does make it sound tedious since it removed the automation aspect i thought the bot was meant to supply?
because what happens if someone updates the entry on the uploads board to fix a error, or to skip a episode (Perhaps the fansub won't sub episode x because it's a recap, it happens). Plus it would make it a easy way to add extra management options for the bot.
In my third paragraph, I pointed out that the bot could be created with the addition of only 1 new Endpoint on the site, which would output New, and Updated information for the bot. This information could contain anything, include deleted series, updated series, and new series. This can easily be combined with Local Information Storage, based on the ID.
While i may not know everything you have planned for the bot, and i've explained some things referencing existing features only, most things i've explained here can be expanded upon to cover any feature you could want, just by being flexible in the way you handle data between the management board/API and the bot.
I think I've covered everything and tried to explain my thought process behind my comment(s). Also my comments are getting longer and longer >.< lol
hanimayahi
commented
8 years ago Is the bot not going to be running as an external separate program that just so happens to consume data from AFTW, and run encodes for AFTW based on that data? Personally i call that an external separate program lol...
Technically yes, but it will be a big part of aftw.
I stated it would be stored in a SQLite Database or a Json file, these would be for the Bot only, and the Bot would retrieve new information from AFTW, then update the storage method chosen, why would the AFTW site be updating the Bots data? Why not just output new information on the Entries for the bot at a single Endpoint, that the bot consumes the new data from and stores it?
I won't even need to save anything locally, this type of data will be regulary fetched (Perhaps updates once per day, and on startup?) from aftw to keep it synced correctly from the management controls. Plus it's not even large data we're talking here, the bot as I'm building it doesn't need to save any data, maybe very minor stuff.
In fact, the code would likely be about the same amount either way you do it...
Yeah, I don't think so.
Especially considering from what i've seen there would be no easy way to just call a function in one of the Management board's classes at an API Endpoint
Why wouldn't it be possible?
I think you are underestimating how much easier it is to change how you consume data, rather than everything because you want to change the way data is handled... Not sure how to explain this correctly, sorry.
Yeah, I don't think I understand what you mean here..
I'm not entirely sure what you are planning that requires the current episode number from the management board, but it really does make it sound tedious since it removed the automation aspect i thought the bot was meant to supply?
How does it remove the automation? What I'm thinking is, that it will read the latest episode uploaded from the management board, and when it uploads a new episode, it updates that value. This will allow us to externally change how and what the bot does, if we would need to.
While i may not know everything you have planned for the bot, and i've explained some things referencing existing features only, most things i've explained here can be expanded upon to cover any feature you could want, just by being flexible in the way you handle data between the management board/API and the bot.
Seeing as we have the option to, official support is better than doing workarounds imo.. It's not about being flexible.
I think I've covered everything and tried to explain my thought process behind my comment(s). Also my comments are getting longer and longer >.< lol
Lol, thanks for your input
Nikey646
commented
8 years ago Yeah, I don't think so. Why wouldn't it be possible?
From my brief 'dives' into the Management board's code, it all seems to be tightly integrated together, and not designed to be apart of a generic area like the API. Hence requiring roughly the same amount of code. Of course, i could be wrong, and the Management board's class system could be entirely interoperable with the Main site's!
How does it remove the automation?
Why is the bot not automatically grabbing the newest episode? Why does it need to read the latest episode uploaded, when it could just internally store the latest episode # it encoded, meaning that it doesn't need to ping AnimeFTW to get the latest episode #, and instead will just need to update the board when a new file is in zeus?
What I'm thinking is, that it will read the latest episode uploaded from the management board, and when it uploads a new episode, it updates that value. This will allow us to externally change how and what the bot does, if we would need to.
I cannot think of the situation that you are referring to where we would need to externally change what the bot does? Unless you are referring to future proofing it for new additions.
Seeing as we have the option to, official support is better than doing workarounds imo.. It's not about being flexible.
I wouldn't call outputting all data on a single EndPoint a work around, since depending on how you design the output, you could easily return small "No new content" messages, allowing more than 1/Day updates, allowing it to start asap on newly added/updated info.
While yes, using the existing Management ajax script could be considered a Work around, It is my opinion, that it is easier and faster than trying to design a new way to use existing functionality.
It seems the main design difference we are both thinking of is whether the data is stored Internally or Externally. In this case, i think it would be better to store the data internally, meaning that changes to the site won't have a lasting negative effect on the bot, since it'll only need to change how it consumes data to get it to the old format that is already implemented. You think it would be better to store the data externally (In this case, on AFTW itself), making the bot rely heavily on AFTW. Both choices have their benefits and downfalls, however in the long run if you decide to stick w/ External data only, then most of my suggestions here are moot ;D
Edit: Woot, my comment was shorter than the previous! ;D
hanimayahi
commented
8 years ago It seems the main design difference we are both thinking of is whether the data is stored Internally or Externally. In this case, i think it would be better to store the data internally, meaning that changes to the site won't have a lasting negative effect on the bot, since it'll only need to change how it consumes data to get it to the old format that is already implemented. You think it would be better to store the data externally (In this case, on AFTW itself), making the bot rely heavily on AFTW. Both choices have their benefits and downfalls, however in the long run if you decide to stick w/ External data only, then most of my suggestions here are moot ;D
Okay, I've been thinking about it since I woke up, and you do have some good points. Seeing as node is asynchronous, I could easily make the bot act as a server, and listen for commands. Storage will be internally, but any changes made from the management controls will be sent directly to the bot when they are made. This way, I can make any controls I want and you get to keep your internal storage ;)
Still, I'd like to be able to update uploads entries through the API, or depending on how complicated I find it using the current ajax scripts. We could put this issue as stalled, and I'll create a new one for the Bot-san controls.
Nikey646
commented
8 years ago I could easily make the bot act as a server
I hope when you say server, you mean it will only operate on LAN, and will not consume data from the internet, because if my memory serves correctly (It generally doesn't when it needs to, like noaw :(), Brad does not wish for the Encoding Servers to be forward facing if possible.
hanimayahi
commented
8 years ago I hope when you say server, you mean it will only operate on LAN, and will not consume data from the internet, because if my memory serves correctly (It generally doesn't when it needs to, like noaw :(), Brad does not wish for the Encoding Servers to be forward facing if possible.
If the servers will be in the same place, yes we can set it to operate on LAN. But, the bot will fetch stuff from the internet either way, nyaa rss feed, torrents??
hanimayahi
commented
8 years ago I updated the requirement.
The API must allow Fay to be able to do the following: