Sell songs by Danny on agora.download

[casey]

This issue could also be in the content repository, where we store the contents of agora.download, but since it might involve changes to Agora and new features, I thought I'd create it here. Feel free to move it though.

I think we should:

• Convert them to MP3s. WAV files are large and the quality difference is not discernible. MP3s are, I think, the most compatible format.
• Add the MP3s to the content repo. This will increase the size of the repo, but three MP3s are relatively small, and we can purge them by rewriting the history later.
• Find a way to exclude them from test.agora.download. I think the easiest way to do this would be to truncate them to 0 bytes when deploying to test.agora.download. That way we can see the files and make sure that pricing is correct, but they can't be downloaded for testnet bitcoin.
• Configure the pricing, according to what we decide to charge.
• Figure out licensing. I think that we should default to something that says "All rights reserved. For personal use only.", unless Danny wants something more permissive.
• Have Danny write an index.md that we can deploy alongside the songs. I almost feel like we should mostly write the index.md files for content, so that they're consistent.
• Deploy!

How does all that sound?

[soenkehahn]

I like the plan. Overall I think I would want to give Danny (and other artists) more control over how to host files. Some more comments inline:

* Convert them to MP3s. WAV files are large and the quality difference is not discernible. MP3s are, I think, the most compatible format.

I agree, and I think that's a good idea. I do however think that this should be Danny's decision. If he wants to sell files in a completely obscure format, we should let him.

* Add the MP3s to the content repo. This will increase the size of the repo, but three MP3s are relatively small, and we can purge them by rewriting the history later.

Yeah, I think this sounds good. Even for the uncompressed .wav files, I would be for this option.

* Find a way to exclude them from `test.agora.download`. I think the easiest way to do this would be to truncate them to 0 bytes when deploying to test.agora.download. That way we can see the files and make sure that pricing is correct, but they can't be downloaded for testnet bitcoin.

Yeah, good idea. I was wondering how to do this.

* Configure the pricing, according to what we decide to charge.

Yup.

* Figure out licensing. I think that we should default to something that says "All rights reserved. For personal use only.", unless Danny wants something more permissive.

Yeah, that sounds right. Where would we put that license text snippet?

* Have Danny write an index.md that we can deploy alongside the songs. I almost feel like we should mostly write the index.md files for content, so that they're consistent.

I kind of disagree with that we should be writing those files instead of Danny. I think it'd be nice to have artists write those themselves. I almost think that the more wildly differing they are, the better. :)

I do wonder how Danny's going to work on an index file though. Does he have to install agora and run it locally to be able to see how the file will be rendered?

* Deploy!

Profit!

Here's one other idea that just came to mind: Could we change our deployment system to host random new agora instances (on kos, connected to the same lnd) that are somehow not publicly accessible. For example they're hosted under test.agora.download/<some cursèd unguessable hash>/files. And they just deploy a local directory to the instance. That way

1. Danny could give us files, including an index file (and possibly included static assets like embedded images) in whichever format is most convenient to him. Dropbox, github repo, zip file, etc..
2. We could easily push new versions to the hidden instance to let him see what it looks like.
3. We don't have to zero out files to test whether the directories are configured correctly.
4. We can test behavior of the real files. E.g. streaming or web players, etc..

This is probably an idea for later though. I'm hoping for Danny we can get away with something easier.

[xasopheno]

You've built such a cool thing. Excited to be a part of the process!

In making this music, we've worked hard to avoid compressing the audio along the way. Mp3 is not lossless compression, so it would be a bit disappointing to see the audio needing to be converted to mp3 in order to host it on agora. I would prefer to sell the files as wav seeing that you can always compress audio, but you can't go back. In my mind, compression should be the choice of the file owner.

[casey]

Hey Danny! Nice to sort of meet you finally!

You've built such a cool thing. Excited to be a part of the process!

Thank you so much! Honestly, I think I'm probably just as excited that you're interested in being involved. Sönke shared your music with me, and I really like it.

In making this music, we've worked hard to avoid compressing the audio along the way. Mp3 is not lossless compression, so it would be a bit disappointing to see the audio needing to be converted to mp3 in order to host it on agora. I would prefer to sell the files as wav seeing that you can always compress audio, but you can't go back. In my mind, compression should be the choice of the file owner.

I think that's totally reasonable. I have many thoughts about this! Some bullet points, in no particular order:

• WAV doesn't have a standard for metadata tags, so we couldn't tag WAV files with the artist, album, track number, title, or embed album art. Tags improve the user experience of playing music a lot, so not being able to do this would mean a degraded user experience for most users.
• From blind A/B listening tests, 175 to 245 kbit/s MP3s are transparent to listeners, meaning that listeners cannot tell the difference between the MP3 and the lossless audio.
• Being able to convert between formats is a strong advantage of lossless audio. Doing a lossy-to-lossy conversion can introduce audible artifacts.
• FLAC is an alternative lossless audio format that supports tagging, as well as having smaller file sizes. The three songs of yours that Sönke shared with me were 151mb total, but 91mb after being converted to FLAC.
• WAV is not a popular format for personal music collections. The popular choices are MP3 for lossy and FLAC for lossless.
• FLAC is slightly less broadly supported than WAV. I think this is only on macs though. Apple is hostile to FLAC for no good reason, and so browsers on macOS and iTunes don't support it. However, the support is still extremely good, as it is an open standard and popular choice for personal music collections.
• I suspect that many music players won't handle WAV well, even if they can play the actual audio, because it isn't popular format for personal music collections.

Given all that, here's what I would propose: We start by hosting the FLAC files for users to download, since they're smaller than WAV files, more popular for personal music collections, and still lossless. Later, we can figure out how to let users download MP3s, at their option. Ideally, we could make it so that a user pays for a track once, and then can download it in whatever format they want. They can download it as an MP3, and then later go back and download the FLAC file, or vice-versa, without paying for it again, which I think is the most user-friendly option.

How does that strike you?

---

Responding to @soenkehahn:

Overall I think I would want to give Danny (and other artists) more control over how to host files.

I think that in this case, we can find something that works for everyone, namely what I outlined above. However, in general, I don't think a policy of giving artists full control over what formats are available or how files are presented would be a good one for us, our users, and ultimately the artists themselves. As I said before, I think we can find a good solution here that works for everyone, but I think it's useful to write about, since we'll probably run into it in the future.

• Consider MySpace versus Facebook. MySpace allowed customization of user profiles with arbitrary HTML, whereas Facebook only allows limited customization. MySpace was very difficult to navigate, because each profile is different. The layout, colors, and buttons on each profile are different, so learning how to navigate one profile doesn't help you navigate a new profile. Facebook is far easier to navigate.

• I'm on two private BitTorrent trackers that have movies. One of them has structured metadata, where each torrent has fields like description, year, screenshots, and thumbnails. The other has a single description field which contains BBCode. The former tracker is far easier to use than the latter, since everything is uniform. If you want to see the screenshots, you know where to look for them, as opposed to them being in a different place for every torrent. Also, I think more movies have screenshots, since uploaders have a dedicated way to upload them.

• A user should be able to expect that if they can play one piece of music by one artist on our site, they can play music by any other artist on our site. If they buy an MP3 and play it, and then buy a WAV file, and then can't play it, that's a terrible user experience.

• I think we'll soon have features that require additional code for each feature we support. For example, previews, adding metadata, reading metadata, and converting between formats. Each format we support is additional code that we'll have to write.

• Formats are different, so, for example, some formats don't allow tagging, so some formats would be likely be second-class citizens in terms of features. WAV is a good example of this. If I add a WAV file to iTunes, tag it, remove it, and re-add it, the tag doesn't persist, because WAV doesn't have a tagging standard (you can use ID3, but it isn't standard), unlike MP3 or FLAC.

• Some formats are basically just worse versions of other formats. For example, APE and FLAC are both lossless audio formats. However, FLAC is more permissively licensed, more popular, and better supported. Barring some compelling reason I haven't thought of, if an artist wanted us to host files as APE, I think we should just say no.

I agree, and I think that's a good idea. I do however think that this should be Danny's decision. If he wants to sell files in a completely obscure format, we should let him.

I think I disagree here. I think Danny is, in this case, being totally reasonable wanting users to be able to get lossless audio, and we can find a good way to make everyone happy.

However, in general, I think there are upsides and downsides to different formats, and we should weigh the pros and cons. Of course, if someone wants to run their own Agora instance, they can use whatever formats they want, but in terms of what we host on our instance, and what formats we add support for, that should be done on a cost/benefit basis, like other decisions we make.

Yeah, I think this sounds good. Even for the uncompressed .wav files, I would be for this option.

Sounds good. GitHub might complain when the repo gets too big, or even earlier if a single file is too big, but we can cross that bridge when we come to it.

Yeah, good idea. I was wondering how to do this.

What if we have some global configuration for test instances? Like you can put test: true in your root .agora.yaml, and if you do that, accessing paid content will return an empty file? This seems like it actually might be easier than doing something custom for our test instance, and would be useful for other people who want to set up test instances as well.

Yeah, that sounds right. Where would we put that license text snippet?

I think we should have it on the invoice page. I think that's where we're saying "You pay us X and you get Y.", so I think legally, and to prevent user confusion, that's where it should go.

I'm going to go buy a physical CD, which I haven't done in years. I'm really curious if there's anything approaching a license anywhere, either on the wrapping or on the case.

I kind of disagree with that we should be writing those files instead of Danny. I think it'd be nice to have artists write those themselves. I almost think that the more wildly differing they are, the better. :)

I think there are a few things to consider:

• For uniformity and security, we should not allow arbitrary HTML, CSS, or JS in index files.
• The .index.md files should help users navigate the site, find things, understand what they're buying, so we need to weigh artistic freedom against that goal.
• Adding the ability to edit .index.md files would be a big feature.

I also think that the scope of an artist-provided description should be limited to just that, the description. Anything about the year, the license, the title, the format, the file size, and the like, should be presented in a standardized way so that it can be easily digested by users.

I think the best option is to have Danny write it, but going forward, I think some kind of self-service option that's limited to basic markdown is probably the best bet.

I do wonder how Danny's going to work on an index file though. Does he have to install agora and run it locally to be able to see how the file will be rendered?

Markdown is pretty predictable, in general. Is there a better way to do that? Maybe just by putting the description into a GitHub issue?

Here's one other idea that just came to mind: Could we change our deployment system to host random new agora instances (on kos, connected to the same lnd) that are somehow not publicly accessible. For example they're hosted under test.agora.download/<some cursèd unguessable hash>/files. And they just deploy a local directory to the instance. That way

1. Danny could give us files, including an index file (and possibly included static assets like embedded images) in whichever format is most convenient to him. Dropbox, github repo, zip file, etc..
2. We could easily push new versions to the hidden instance to let him see what it looks like.
3. We don't have to zero out files to test whether the directories are configured correctly.
4. We can test behavior of the real files. E.g. streaming or web players, etc..

This is probably an idea for later though. I'm hoping for Danny we can get away with something easier.

I think all the goals are good, but that sounds like it would be a lot of work, and would make our deployment system much weirder, since it would have to support a bunch of sub-instances. I talked a little bit above about having an option that prevents an instance from actually serving paid file content. That seems easier, but has the downside that we can't test the behavior of real files. One way to solve that would be with an option that would make an instance only serve previews instead of full files. So with that option set, an instance would only return the first 30 seconds, for example, of a song. That seems easier, since we want to support previews anyways, and would be useful for anyone who wants to run a test instance.

One downside of this is our test instance is publicly accessible, so people could see things before they went live, but we could fix this later with some kind of permission system, which we might need anyways. (I.e. certain content is only visible if you log in, subscribe, are an admin, etc.)

[xasopheno]

Great to meet you as well. "Long time listener, first time caller." You have great repos!

FLAC sounds great. Perhaps we could host both the FLAC and Mp3 files as well. Might be nice to offer both.

I would like to be able to write a README about the work. It would be great to be able to embed other types of files in the README - particularly images. Video and audio might also be nice to embed. I like the idea of some standardized information (maybe via an associated toml file?), but in general I would prefer to present my work in whatever way I might see fit.

On the topic of allowable file types, I wonder if it might be helpful for me to share my thinking...

As a person that makes a lot of different kinds of things, I'm excited about Agora as a possible place for me to host any of the weird things I make regardless of the file type.

• High resolution audio - Like the audio we've been discussing
• Low resolution audio - Phone recordings from rehearsals/shows.
• Video - https://vimeo.com/manage/videos/229055691
• Binaries - I've been working on some art programs that would probably be best distributed as a binary.
• Text - What if I accidentally wrote a poem?
• Images
• Code/Python Notebook
• Trained neural network
• etc.

To me, it feels like an exciting, weird, and wild time, where new mediums are being created all the time. And given that as the current landscape of thing making, I'm most inspired by an open space that gives me room to express the new and weird things I'm excited to share.

Of course, it might not be the case that Agora is a system that can or should handle selling all of these things!

All of this said, I think the simplest system that would be the most useful for me would be a system in which I could serve a Folder of Files with an associated README and a LICENCE, I could probably stretch that to be useful for most anything I'm currently working on.

Hope some of this is helpful. I'm super excited about Agora and grateful for all the work you two have done to get it working!

[casey]

Great to meet you as well. "Long time listener, first time caller." You have great repos!

Thank you!

FLAC sounds great. Perhaps we could host both the FLAC and Mp3 files as well. Might be nice to offer both.

I that would definitely be ideal. It would be nice to have support in Agora for auto-transcoding from FLAC to MP3, so they don't have to be kept in sync, but not having that functionality shouldn't stop us from hosting the MP3s.

I would like to be able to write a README about the work. It would be great to be able to embed other types of files in the README - particularly images. Video and audio might also be nice to embed. I like the idea of some standardized information (maybe via an associated toml file?), but in general I would prefer to present my work in whatever way I might see fit.

I definitely want you to feel like you're presenting your work how you want to present it.

My concerns are all around making the site navigable and understandable for the user. (Which could also translate into more sales for artists and more eyeballs on their work!)

I think the tension is around the question of is Agora, and is our Agora instance in particular, like GeoCities, where users can create home pages with arbitrary content, and additionally charge for some parts of those homepages, or is Agora like Bandcamp, where artists have more limited freedom in customizing their pages, but where it's very easy for users to navigate around, find what they're looking for, etcetera.

I really hope we can thread the needle and do both.

On the topic of allowable file types, I wonder if it might be helpful for me to share my thinking...

As a person that makes a lot of different kinds of things, I'm excited about Agora as a possible place for me to host any of the weird things I make regardless of the file type.

• High resolution audio - Like the audio we've been discussing
• Low resolution audio - Phone recordings from rehearsals/shows.
• Video - https://vimeo.com/manage/videos/229055691
• Binaries - I've been working on some art programs that would probably be best distributed as a binary.
• Text - What if I accidentally wrote a poem?
• Images
• Code/Python Notebook
• Trained neural network
• etc.

I definitely want to support all of those things, and files that you just download to your computer will always be an option.

For specific kinds of media though, I think that Agora should be opinionated, and our instance at agora.download should be even more opinionated.

• Audio: There's very little reason to deviate from FLAC and MP3 for audio.
• Video: Every video should play in the browser. If a user can play one video in the browser, but then buys another video and it can't play in the browser, that's a really bad experience.
• Text: We should only allow UTF-8, anything else is just a mess from a technical perspective.
• Images: It's hard to justify anything other than PNGs and JPGs. Even GIFs are hard to justify, since they could just be videos, which have lower file size and higher quality.

I would never want to prevent some particular piece of art from being shared on Agora, but I do want our users to have the best experience, and I do want the content on Agora to be available to the most people by virtue of using common formats.

To me, it feels like an exciting, weird, and wild time, where new mediums are being created all the time. And given that as the current landscape of thing making, I'm most inspired by an open space that gives me room to express the new and weird things I'm excited to share.

Of course, it might not be the case that Agora is a system that can or should handle selling all of these things!

I think it can be! It'll just take some work to thread the needle and balance all the concerns involved.

All of this said, I think the simplest system that would be the most useful for me would be a system in which I could serve a Folder of Files with an associated README and a LICENCE, I could probably stretch that to be useful for most anything I'm currently working on.

To give you some background on where I'm coming from, I've been a BitTorrent user for a long time. BitTorrent is easy and flexible, just point a torrent client at some files and you can share them with others. However, that same flexibility creates an ecosystem with severe usability issues. There are a plethora of random formats, there is no standardized metadata, nothing is machine readable, every download requires manual, technical user intervention to unpack and play. BitTorrent is now used only by the most dedicated and technical of users. I don't want Agora to suffer that fate, because I don't think it will be as impactful and as useful as something that works hard to avoid those problems.

Hope some of this is helpful. I'm super excited about Agora and grateful for all the work you two have done to get it working!

It's super helpful. And I think we can work to find solutions that make everyone happy.