Assets should only play for correct tag

[seeReadCode]

Chris's plate recording plays with tags that it shouldn't be associated.

From Halsey:

I checked on that “plates” asset that continually plays. There wasn’t anything particularly weird, but it did have the plate tag as well as the kitchen tag, the latter being unnecessary due to tag relationships. I removed the kitchen tag, so if you can let me know if the behavior is improved, that would be great. That said, having the kitchen take shouldn’t cause that problem, so if the tag removal does make a difference, we’ll need to investigate why. Probably a server thing…or a data thing with tag relationships etc.

[hburgund]

We should investigate what the defaults are for the initial listen state as well. When the stream is initiated, the listen tags marked with default=TRUE should be sent with the request (or shortly thereafter with a PATCH?) so that the stream knows what to include right off the bat. If no tags are sent, the stream in theory should only play the background audio/music, though I can't say 100% that the server behaves this way currently as no tags might default to all tags.

In any case, can you have a look at what tags are sent initially as if the Plates tag is sent, that would help explain this a bit.

[seeReadCode]

If memory serves me, the first tag in a room is selected (i.e. PATCHed) when the room is loaded. I think my logic there is a bit more concise than using a default attribute especially given that users would expect a linear flow of assets as per the UI.

If you want to insist on the default logic, please make that a separate "enhancement" issue.

[hburgund]

OK, we can leave it as you have it for now, but knowing this, I am guessing that this is why the Plates comment is played first often. Or at least is a contributing factor. I will think about a better overall solution as I don't think this one is going to be flexible and intuitive enough in the long run.

Given that there is a physical representation of the content ie the actual rooms, I don't think that users will have an expectation of the content following the GUI necessarily. Also, the default approach gives more flexibility by using server-side controls, so we may want to use that.

My gut says that the best solution might be to generate the stream with no assets initially and wait until either the user manually selects a tag or the automatic location system sends a location PATCH. We may need to present a GUI hint to let the user know to select something, but that is probably better than always having certain assets play first regardless. Like I said, let's not worry about this right now and we can continue to think about a better solution. Beta testing will be helpful as well.

[seeReadCode]

Well, the contribute page requires a tag to have been selected, so I'll need to manage the contribute button's enabled status depending on whether a tag has been selected (whereas before I could assume one always had been).

I take your point with this default logic. Here's what I'll ask if we go ahead and go in this direction: I had no idea that this attribute existed. Maybe there was some doc on it somewhere across all the docs that are floating around but if I had known this was what you wanted, then I could have planned for it and built out my logic accordingly but you presented it in this thread above as though I should have been aware of it's existence and I simply wasn't. If you want to make the server as responsible as possible for the client UI, I need a much more explicit/confident understanding of the data that comes from the server and that is needed by the server.

As discussed Friday, we are going to want to tighten up the whole UX on this interface and rather than hit road bump after road bump that will require me to re-code again and and again, can we please go ahead and articulate the whole spec as you would like it to be? I'll start it here but a Google Doc might make the most sense.

I'm glad to help but I like to avoid burning hours building things that don't fit the functionality you have imagined. So let's imagine, I've started the session and that's it. Please answer the following with regards to the UI, server calls, and what I should expect back from the server? Please reference all necessary data from the server and it's sources.

What happens when I load this page?

Then, what happens when I click play?

What happens when I click a tag?

What happens when I click that same tag?

What happens when I click next?

What happens when I click previous?

What happens when I switch rooms?

What happens when I load a map?

What happens when I background the app?

What happens when I resume the app?

What happens when I lose my connection?

What if I am not on wifi?

What happens when someone locks their screen?

What happens when I walk around?

What happens when all the assets in a tag complete?

What should the UI while waiting for an asset to start playing back?

What happens while the stream is playing, i.e. what is the purpose of the heartbeat? What data should it include or not include?

What happens when I tap contribute?

What should the time on the left of the UI show and where should that time from?

The one on the right?

Anything else that should be specified?

---

Sorry if that seems like a bit of a big ask but I just want to avoid further misunderstandings and burn here. Thanks!

[seeReadCode]

The immediate issue here seems to be resolved as of testing today.

[hburgund]

good news...I will respond to your earlier request as soon as I am able. It will be good to get it all written down, even if a bit late.

This makes me think that we/I may have made the wrong decision as to where to track issues for this project as all of this good info should really be kept in the primary repo, not your fork, since much of it is broadly applicable. Oh well, I can copy stuff over manually at a later date when everything gets merged up, unless there's a better idea...?

[seeReadCode]

Re: issues and repos – I need a higher level of access on the main repo in order to access issues I think.

When we want to copy issues over, this should be handy: https://github.com/google/github-issue-mover

[hburgund]

Here are some initial responses; likely there will need to be more ultimately, but this should get things started.

I am not going to answer all of the questions now because there are numerous places where as best I can tell, everything is working fine and since I don’t know what is actually happening in the app, I don’t want to guess and be wrong and cause issues as a result. I’m happy to discuss as needed to give you the further info you require.

What happens when I load this page? Seems OK to me as is.

Then, what happens when I click play? POST api/2/streams/ send latitude and longitude only if geo_listen_enabled=TRUE Client waits for server response and plays audio from returned stream_url when received If no stream_url is returned, client error should be displayed. We can talk about those details. I think ultimately after the audio starts playing on the client, a PATCH api/2/streams/ call should be sent containing the default listen tags. For now, we can stick with the logic you built in which, I believe, sends a PATCH call with the first tag in the GUI display.

What happens when I click a tag? PATCH api/2/streams/ with the new tag id as parameter Continue playing audio stream with no pause; the current asset will fade out and a new asset for the new tag will fade in shortly thereafter. Currently the fade-out doesn’t work, but will soon.

What happens when I click that same tag? I haven’t really considered this much, but if the active tag is clicked again, I don’t think anything should happen since it is already active. If another tag has become active in the meantime and the user wants to go back to the previously listened tag, the same PATCH call should be sent. I don’t think the server would care if the PATCH call was sent even in the active tag situations, but I’d have to investigate.

What happens when I click next? POST api/2/stream/:id/next NOTE: this is not currently functioning on server, but will be soon

What happens when I click previous? POST api/2/stream/:id/previous NOTE: this is not currently functioning on server, but will be soon

What happens when I switch rooms? PATCH api/2/streams/ with tag id for new room as parameter

What happens when I load a map? I have no idea what people want with the map screen. Personally, I think it doesn’t make sense to have a map unless we are using interior location, but for some reason, other folks want the map. My understanding is that they want a full-screen vector map of the site loaded that can scroll and zoom. I will investigate.

What happens when I background the app? My understanding is that there are several ways an app can be backgrounded (screen dimming, other app opened, phone/text/etc interrupt…maybe others?), but I don’t pretend to understand the intricacies of iOS in this regard. I think either way, the app should continue to function i.e. playing audio and updating the stream as necessary, unless it is an audio interrupt in which case the audio will stop playing and the stream should be paused just as if the Pause button was pressed. Likewise, when the audio interrupt is over i.e. phone call ends, the stream should be un-paused.

What happens when I resume the app? I think the above answers this?

What happens when I lose my connection? If connection is lost for long enough that the stream runs out of buffer, a popup notification should display telling the user that audio streaming is only possible when a data connection is in place. The LISTEN functionality should be disabled and the button disabled as well to minimize confusion. The app can still function in SPEAK mode as Roundware can record assets, store them locally and upload them once back online.

What if I am not on wifi? I don’t think wifi vs. data connection makes any difference?

What happens when someone locks their screen? see backgrounding response above. App should continue to play and update stream.

What happens when I walk around? Nothing if geo_listen_enabled=FALSE but if TRUE, every time a new position comes in via CoreLocation, a PATCH api/2/streams/ is sent with the new latitude and longitude as parameters.

What happens when all the assets in a tag complete? This is something that we were hoping to get feedback on from testers. My inclination is when geo_listen_enabled=FALSE the app PATCHes the stream with the next tag in the GUI, if one exists. When geo_listen_enabled=TRUE nothing should happen; the app should wait until the user moves to a new location where there is more unheard content.

What should the UI do while waiting for an asset to start playing back? The UI should display a spinner when waiting for the stream_url initially or while the stream is buffering when returning to listening after contributing. As far as assets in the stream go, I think the GUI should reflect the current or most recent asset played by highlighting that tag GUI element. So when waiting for an asset, it shouldn’t change from the last asset played until the new asset starts.

What happens while the stream is playing, i.e. what is the purpose of the heartbeat? What data should it include or not include? The purpose of the heartbeat is to keep a stream alive when it is likely to be used again while ensuring that streams don’t continue to exist after they are no longer needed. The server will automatically kill streams if there is no activity on a stream for a period of time (default is 200 seconds, I believe). The reason for this is that stream generation is resource intensive on the server so we don’t want to waste effort keeping streams going that are no longer needed.

The heartbeat is a benign method for telling the server that activity is still happening on a particular stream so it should not be killed. Stream “activity” is defined as any api calls involving the stream’s id or active listening of the stream (as tracked by Icecast). So there is no need to send heartbeats when the stream is being listened to, but when the user is contributing or the stream is paused, we want to send heartbeats periodically to make sure their stream is alive when they are done and want to listen again. When the user quits the app, no more heartbeats are sent and the stream is killed after 200 seconds.

Due to my lack of knowledge on app backgrounding, I’m not sure if we will want to consider some more detailed control here, like if the app is backgrounded and not being listened to, don’t send heartbeats because that probably means the user is done…?

What happens when I tap contribute? this flow looks pretty good to me in the current app...

What should the time on the left of the UI show and where should that time come from? Which screen? Listen? I see the two counters on that screen in the app, but in my screen flow pdf, I only see one counter. My understanding was that this was supposed to count down the remaining time for the current asset. This was a topic of some debate during the design process, so we were going to wait for some testing feedback before implementing something here, as best I recall. If it is a countdown time for the current asset, we would need to have the server provide you with the length, which we could do in the stream metadata, most likely.

The one on the right? see above

Anything else that should be specified? I’m guessing there is more, but for now will let you follow-up with more questions unless I think of anything to add.

[seeReadCode]

Other questions/tasks for later moved to #20 #21 #22 #23

[seeReadCode]

I'm closing this issue and will move the above tasks off to other issues. I'll move Halsey's answers into a markdown file for the time being as well.