Channels (groups) management

[kewalsk]

Is somebody working on that? This would be the great improvement of this application. Already mentioned about it in original project #https://github.com/Pmmlabs/OpenPeriscope/issues/122

[gitnew2018]

I want to add it but there is nothing about groups in http://static.pmmlabs.ru/OpenPeriscope/ so i don't know how to even access them by API.

[kewalsk]

Yes it is not documented but no problem. I can share my knowledge.

[gitnew2018]

Heh, can you share how to obtain "the knowledge", without asking people like you ^^.

[kewalsk]

Yes I know 2 ways to do that. First is to use wireshark to catch the network traffic beetween the client app and server, but I don't know how to handle https. Second is to decompile android client and get the api implementation from sources. It require some basic Java knowledge but the code is easy to understand. This is the wey I'm doing this.

[gitnew2018]

Thanks, I'm not going to decompile android client anytime soon 😁. Just tell me what are the API calls for groups.

[kewalsk]

I will create the description on github wiki or somewhere else soon.

[Max104t]

Yes I know 2 ways to do that. First is to use wireshark to catch the network traffic beetween the client app and server, but I don't know how to handle https. Second is to decompile android client and get the api implementation from sources. It require some basic Java knowledge but the code is easy to understand. This is the wey I'm doing this.

Periscope was not happy with proxy settings + fiddler. a wiki on how to use wireshark would be great!

[kewalsk]

Yes, finally.

The documentation is more or less ready. Please take a look on it on my wiki.

[gitnew2018]

Thank you, Bugs have higher priority than new features and unfortunately I'm procrastinating right now 😞. So it might take a while, unless someone wants to help and do it.

[kewalsk]

I will help for sure.

[Max104t]

I like to review this with you before I submit the pull request.. How do you like Accordion style in the Edit section?

[kewalsk]

I think it's good idea. There is a lot of settings already. Maybe it would be useful also to separate periscope settings, which needs to call api on save, from just saving the local parameters? It's not clear if saving is required on changing some My-OpenPeriscope settings.

[gitnew2018]

I think groups need their separate section. I agree with @kewalsk on the other points.

[Max104t]

testing and adding join/leave actions :-D

I need the refactor for splitting the code to get it done..

https://github.com/Max104t/My-OpenPeriscope/tree/support-groups

[gitnew2018]

Ok, I'm going to review and do some basic tests of your PR.

[gitnew2018]

Turns out I'm in 1800+ groups 🤩

[kewalsk]

Yes, we will need a function like delete_me_from_all_shitty_groups_and_never_let_add_me_again

[Max104t]

I've added :

• group list
• group images
• group broadcasts ()clicking on group name
• owner / invitee link

TODO:

• members count => click will show members
• I need help REST API to accept / reject, and leave groups?

latest on https://github.com/Max104t/My-OpenPeriscope/tree/support-groups

[Max104t]

Yes, we will need a function like delete_me_from_all_shitty_groups_and_never_let_add_me_again

I like to have this as part of ML plugin.. I have a database of more than a million users tagged and refreshed daily.. it even groups recreated profiles matching description + user names to existing one.. with 2 degrees of separation..

I like to use this data to rank groups and detect members interests..

[kewalsk]

That would be great to connect some external database like MySQL to store locally the users, groups, broadcasts, dowloaded videos with additional attributes.

I didn't yet experimented with accept/reject invitation, I suppose the patchChannelMember should be used.

The group leave is quite easy: call the deleteChannelMember with your user id.

[gitnew2018]

Branch merged to master. I'm going to work some more on it. I want to add some kind of preloader circle. Show all groups and invites because right now it shows only ~1860 for me. Maybe some kind of pagination. View group members and history. I'm thinking about iframe or popup for that.

[kewalsk]

I'm working on the group members and history.

This will take some time because of my javascript experience or rather lack of it, but we have to agree about who is developing which part to avoid conflicts.

When the group list is big (my record is over 5k groups) the critical point is the ease and the response time to show and manage list of groups and then the list of members. I'm not going to mix the invites, groups and membership so there will be 3 main menu items and the functionality will be based (similar) to the list of broadcasts or list of users. The chunks are the open issue - I was thinking about just the button "get more" to retrieve the next part if there is any and append it to the list. The same issue is also related to the members and history and for a big groups (a lot of members) there is even no real possibility to get all parts of history - it is simply too long and the time to get it probably would take several hours (I didn't actually tried to get all the history if the number of chunks were over 2k). The more professional solution would be to cache locally in some kind of database all retrieved information, compare if something changed, retrieve only changes and show the list of data from the database using CRUD with paging, searching and so on.

[gitnew2018]

Ok, you can do group members and history. It's fine by me 😁.

Separate menu sections make sense for huge amounts of groups and members. The "get more" approach seems much simpler to implement than pagination, so we can go with that, at least for now.

I was going to ask why would anyone need to see such long history, but then I realized that, why would you even want group support here, if it wasn't more powerful than official app 😏

[kewalsk]

My goal is to have it more useful than official app, especially when it comes to browsing and searching the group members or history - currently not possible in official app. Also interesting would be sorting groups by any field: name, creation date, activity, owner, adder. Making statistics from history by type of action.

It looks like a periscope team doesn't pay too much attention to this part of application functionality. The all necessary data is there but for example the history list in official app is something like a joke. Maybe in future they will do something with it, but so far the things are rather going worse.

[Halinks]

Are you still there @kewalsk ? I love the way you can browse groups in OP.. probably one of the best additions so far!!! Are you working on coding the part where you can see who's in the group? I would love to see that feature 🙏

[kewalsk]

Yes, still working on that as much as I have a time.

The group table is almost ready, except the sorting. Maybe I will add also paging in future because when there is a lot of groups then the building the html table takes more time than calling the api.

The list of group members is ~50% done but not commited yet.

If you want to test it please take my branch groups. It is not ready to be merged into master yet.

[Halinks]

Looks good! I don't see any bugs so far.

[gitnew2018]

There needs to be something done with the very poor performance. It's blocking whole app for couple of seconds. Quick ideas:

• abandon the table
• make it asynchronous example

[kewalsk]

Please describe the problem in more details. Which part has the performance problem? All the API calls are asynchronous or you found it to be synchronous somewhere? Is there any other operation you want to to make asynchronous?

The biggest issue that could make poor performance is appending the long list of groups, members or history actions. Abandoning the tables and making the simple list instead will not give any difference. The loop on the > 2000 groups or other objects to create a list takes long time. The most reasonable solution I can see is to make paging to display only small part of the data. I will go for it some day but now I'm concentrated on the functionality not on the performance.

There could be some other solution that could work but I don't know how to make it - appending only fist small part of the list and then, on response to some event as the user scrolls down, append more. If you have seen something like that implemented just let me know.

[gitnew2018]

When you compare "appending the long list of groups" in existing groups implementation it takes like 2s and in yours 6s, my initial impression is that html tables are much slower.

Images in this app have lazy load implemented(lazysrc) you could check this out.

[kewalsk]

How many groups you have?

[gitnew2018]

Around 2.5k but displays around 1750 in first batch.

[kewalsk]

I don't know how did you get 2s. I've made additional logging and the results are 5s in Groups and 9s in Membership, but the Groups list is not showing all information and therefore has less information to display (the decoded user is not implemented yet) and they are limited to first ~1700 batch.

3340:11480:1119/213213.693:INFO:CONSOLE(120)] "GroupsController.load_group_membership 1681 groups, START at 2019-11-19 21:32:13", source: chrome-extension://openperiscope/Groups.js (120) [3340:11480:1119/213218.664:INFO:CONSOLE(133)] "GroupsController.load_group_membership 1681 groups, ENDED at 2019-11-19 21:32:18", source: chrome-extension://openperiscope/Groups.js (133) [3340:11480:1119/213237.520:INFO:CONSOLE(70)] "MembershipController.displayData 1681 groups, START at 2019-11-19 21:32:37", source: chrome-extension://openperiscope/Membership.js (70) [3340:11480:1119/213246.206:INFO:CONSOLE(77)] "MembershipController.displayData 1681 groups, ENDED at 2019-11-19 21:32:46", source: chrome-extension://openperiscope/Membership.js (77)

Technically the table and the list is only html tag name difference (div or tr). There could be some overhead when the engine is displaying the output, but for the html document preparation the difference should be minimal. On the other hand looping to create ~1700 rows is something that shouldn't take 6-9s. I suppose the same operation in Java would take less than a second.

If you don't like the tables please feel free to implement different display options. I was thinking to make additional checkbox: "display as table/display as list" but I have already spent too much time to make UI look reasonably good or good enough. Anyway, reducing the time from 10s to 5s is not enough.

[gitnew2018]

You're probably right. Don't worry about it I will accept PR even if it's slow. Focus on functionality is the right approach. Looks and performance can be dealt with later.

[kewalsk]

Regarding the lazy load, this is done only for loading images. I didn't found any other solution for this issue so far except the advises to place "load more" button at the end which is not very useful. I have such a button already to get the next batch of ~250 members or actions from the server.

The other performance killer is decoding the user id. This can be done for more than one user at once by /api/v2/users but the upper limit of this call is 100. The information about user is then forgotten and the call is done over and over again. I want to make some kind of local caching of user information to not get the same user info from the server until expired after some defined time. This will need some local database to store users and maybe also other information like groups members and history. Do somebody have any experience with connecting to databases like postgreSQL or mysql from NW.js apps? The other solution I'm thinking about is a micro-service architecture to do the work and deal with the database on some external server.

[gitnew2018]

Lazy load is for images but it is triggered by scrolling, maybe similar approach could be used to load next batch. window.sessionStorage as cache?

[mirkosergi]

Hi all, do you know which API is called when you from the app want to add a new member to a group but you don't select one or more members while you click on the "Invite Friends in the group" icon the one looking like Telegram logo? This is useful to me because you get the invite to group in the format https://b.pcps.live/xxxxxxxxxx

I didn't manage to find any reference around, not even in kewalsk wiki (https://github.com/kewalsk/My-OpenPeriscope/wiki)

Thanks a lot