sl-service-account
commented
5 months ago JIRAUSER341305 commented at 2024-01-10T04:17:50Z, updated at 2024-01-10T04:25:19Z
Admittedly, I've done very very little sound stuff, but my understanding is that SL uses a fixed sample rate, so frames, start/end times, etc., is just basic math with known numbers. So why not just have the start and end in samples:
key llSoundFragment(key sound, integer start, integer end)If the sample rate may ever change, then we just need a way to get it — could be handled just fine by BUG-232877 accepting keys as well as inventory names, and returning both length and sample rate.
There's also no need to save the extracted sample in script memory, or on the sim at all — just need to store the originating UUID, and start/end sample indexes (grand total of 24 bytes per fragment), then send them to the viewer when called for. This has the added benefit that slicing a sound into multiple parts, the viewer only needs to download the original sound sample once.
The script could be limited to some number of sound fragments, similar to listens, you would need to release them when they're no longer needed (128 should be enough for anyone!). I'd think this sample registry should probably be object-bound, but that doesn't necessarily preclude the keys being available region-wide as you suggest (the sim just needs to maintain an index of all the active fragment keys — though a function like llLoadSoundFragment that does a more object-to-object binding could still work). And in place of a lifetime, just drop the fragment. Allowing a script to generate several instances of the same fragment would suffice for lifetime management.
How would you like the feature to work?
The sound functions such as llPlaySound, llTriggerSound, llLoopSound, etc, should be able to play a specified fragment of a sound clip.
It is quite a number of the sound functions already and more may come in future. All of them receive the name or key of the sound clip to play as parameter. We use this fact and can avoid creating a new function for every of them. Instead we have only one additional function:
This function receives a sound clip, given either by sound in inventory or by a key and returns a key, which will be a 'reference' to the provided sound clip and the time fraction to play.
The returned 'fragment key' can be then used within the sound functions which expect the sound name or key as parameter.
The parameters frames, start, end define the position of the clip fragment:
{}frames > 0{}: The parameter frames gives the number of frames the sound clip is subdivided into, the parameters start and end give the first and last frame to play. For example if the sound clip length is 5.5 seconds, the definition (55, 17, 17) references the 17th frame which is located at the time position of 1.7 to 1.8 second.
{}frames = 0{}: The entire duration of the sound clip is mapped to the interval of [0.0, 1.0] seconds (inclusive) while the parameters start and end give the position in this interval. For example when the sound clip length is 5.5 seconds, the definition (0, 0.1, 0.2) references the fraction of 0.55 to 1.1 seconds of the sound clip.
{}frames < 0{}: The parameters start and end give the absolute values of the start and end positions of the sound clip.
The negative positions count from the end (or from the last frame). The parameter start = 0 means the start of the sound clip, the parameter end = 0 means the end of the sound clip.
A special example: The definition (-1, 0, 0) makes the entire sound clip to a fragment to play and is used as an 'alias' for the sound clip.
Other negative values for the frames parameter can be also defined.
Properties for the fragment key and reference information.
Following properties are required:
The fragment key can be used again for creating an other fragment.
The fragment information (the referenced sound clip and positions) are saved locally within the script (using the script memory) and will remain there as long the script did not reset or not purged the sound fragment from the memory (the function for this is introduced later).
A script can create and hold multiple sound fragments at the same time.
The fragment information will not be stored into the asset database. When the sound fragment sharing will be not implemented (the next section), there is no need even to keep it within the region memory, so that this information remains entirely within the memory of the script created the sound fragment.
Sound fragment sharing
For more applications, following properties are highly preferable to have as well:
The fragment information would be cached within the region (the cache size should keep a reasonable amount of created fragments.)
The fragment information would be shared between scripts, even within different devices. For example issuing {}llTriggerSound{}(fragment, 0.0) would provide the fragment information to the called script and it will remain there as well, even after the region cache lost the fragment information already.
A special function for transferring sound fragment to an other script can be invented, too, for example:
The function is called by the script receiving the sound fragment information, while the fragment key is provided by the created script, e.g. via the private chat. This function avoids playing the sound fragment with zero volume by viewers in range in order to make the script to fetch the fragment information.
The return value is TRUE if the provided key is a valid key of a sound fragment and the fragment information is available in the script memory or could be loaded. FALSE when it was lost (purged from the script memory for example.) When the sound fragments will have a lifetime (more in next section) the return value will be the amount in seconds until the sound fragment is purged from the script memory by lifetime process. Unlimited lifetime will be returned by -1 in this case (a special negative number). This way the function will be also used to test if the fragment key is valid and not only for receiving the sound fragment from an other script.
Note: I separated the fragment sharing as a sub-feature because i am not sure if it can be implemented in an optimal way, but omitting this the feature will lack interesting applications.
Lifetime
When sharing of the sound fragments is implemented, the function llSoundFragment may need an additional parameter:
The parameter lifetime, when positive, provides the time in seconds after creation, after which the sound fragment is removed from the script memory. When 0 or negative, the fragment key remains valid and the sound fragment is stored within the script memory until reset.
Purging of the sound fragment memory.
To manually remove the sound fragment from the script memory there may need to be an other function:
When fragment = {}NULL_KEY{}, all sound fragments cached within the script memory are purged.
Why is this feature important to you? How would it benefit the community?
This feature allows to create a number of new applications. Especially after long sound clips can be uploaded:
Sound instruments and sound fonts:
Creator uploads a sound file as a sound font: It contains all notes within specified frames.
The music instrument triggers or plays sound fragments (the notes) either in a predefined order or on control of the user.
The same or various creators can upload more sound fonts for the same instrument using the same frame subdivision.
The user can choose the sound font to play.
Collected device sounds, instant playing:
Various sounds for a device (door sounds, key sounds, door bell, etc.) can be uploaded in a single file.
Device triggers required fragments.
Once the viewer has downloaded the sound clip, all required fragments can be played instantly, minimal sound latency since the viewer already has the file.
Cropping of sound clips:
Creator uploads sound having a copyright or other sequences at the beginning or/and end of the sound clip (e.g. loud noises or disturbing phrases etc.)
In regular use, a script creates a sound fragment excluding the extra positions.
A device using the sound as demo will play the full sound clip and play extra noises as well.
A device using the sound clip as a regular product will play the sound fragment that has no extra noise in it.
When the sound is played via UUID taken/stolen from the viewer cache the extra noise will disturb as well, the sound clip requires extra cropping before using it.
When scripts can share the fragment keys this also allows more applications, for example:
Different scripts in different prims or objects can play the same sound fragments synchronized (after requesting the sound fragment from a script created it).
Demos for a sound clip or sound font:
A music instrument (like exampled above) requests a sound font demo from a vendor.
The vendor provides an alias, which is the entire sound clip in a single fragment with an other key.
The music instrument preloads it via llLoadSoundFragment and contains now the sound fragment in the script memory.
The music instrument can play it for specified time, then purges or the sound alias is invalidated automatically.
Using an alias for a sound clip allows to hide the actual sound key from other scripts. The amount of sound aliases a script can hold is limited by the script memory, even if the aliases have unlimited lifetime. Hence there is no danger that the region will be overthrown by excessive use of this feature.
Original Jira Fields
| Field | Value | | ------------- | ------------- | | Issue | BUG-234965 | | Summary | Proposal to define and play a fragment of a sound clip | | Type | New Feature Request | | Priority | Unset | | Status | Accepted | | Resolution | Unresolved | | Reporter | Jenna Felton (jenna.felton) | | Created at | 2024-01-09T13:57:55Z | | Updated at | 2024-01-24T19:29:26Z | ``` { 'Build Id': 'unset', 'Business Unit': ['Platform'], 'Date of First Response': '2024-01-09T22:17:50.081-0600', 'How would you like the feature to work?': 'The sound functions such as llPlaySound, llTriggerSound, llLoopSound, etc, should be able to play a specified fragment of a sound clip.', 'ReOpened Count': 0.0, 'Severity': 'Unset', 'Target Viewer Version': 'viewer-development', 'Why is this feature important to you? How would it benefit the community?': 'This feature allows to create a number of new applications. Especially after long sound clips can be uploaded:', } ```