Using uppy without the UI

[oyeanuj]

Hi @arturi @goto-bus-stop @kvz! I am trying to integrate uppy into my existing React (with Redux) app, following the examples mentioned here by @arturi and here by @goto-bus-stop. I had two questions that I think are related to #116 but probably warrant their own thread.

1. Playing around with the current way of using Uppy made me want a little bit more finegrained control over the upload process. Currently, the Core module seems to be a blackbox and holds all the state for the uploads. I am wondering the team's current point of view on the balance between exposing complexity vs blackbox approach of the current Core module? The core module instead of keeping state could expose the events, and utilities for the developer to use? Not dealing with state might make it also less daunting for a developer to be able to add/patch/fix the core module. And then I did come across this (abandoned?) library https://github.com/hedgerh/uppy-base which seems to have some of those ideas. In general, in the React world (hence, my bias), stateless components pattern work well.

2. If using without the UI, is there a way to import only the core module and the plugins being used? I feel like having these as separate packages that one could use would make the load much lighter and make the project more modular as well.

Again, apologies if these issues have already been discussed but felt important enough to bring up in this ticket.

Thank you, all!

[richardwillars]

Just to add to the conversation.. as a proof of concept I just got this working in a React Redux world (with the idea of not modifying the existing core at all).

I simply created an Uppy plugin which then calls a Redux action every time the state changes. This then allows for a React UI to be built based on the Redux store. Although there's now two sources of truth they stay perfectly in sync and the Redux store is read-only (all modifications go through the Uppy event system).

https://gist.github.com/richardwillars/857e845cbedc6695ba8e1861b4b1e50e

So far I'm able to add photos, upload them and also show a list of their thumbnails (with their upload progress, states etc). No issues so far!

[arturi]

@richardwillars Wow, that looks super rad, thanks! Would you mind doing a PR for this plugin, so we could try and it and merge if all is good? That’s actually something we’ve been discussing a few times: allowing easy state syncing between Uppy and Redux.

But yeah, I’m also wondering if there is a way to then sync things back to Uppy from Redux actions, maybe we should also provide reducers like addFile and upload that treat Uppy like an external API that they call, and then also return new Redux state.

[arturi]

I am wondering the team's current point of view on the balance between exposing complexity vs blackbox approach of the current Core module? The core module instead of keeping state could expose the events, and utilities for the developer to use?

Well, the thing is, if you want simple uploads, you could use good old Fetch and XHR. If you want tus, you can use low-level tus-js-client. Same goes for everything else, excluding uppy-server stuff, but we could expose more APIs and then its also usable like this.

And then Uppy is more like a glue that ties uploaders and UI together, normalizes things like progress in files, updates from encoding backends and things like that.

I am not sure how we could keep it easy to use for everybody, if we allow managing your own state in Redux, or MOBx, or file object, without duplication (having two sources of truth). We can’t make sure file object will have the same shape, like progress, preview and type properties in it. So we’ll loose that normalization that you currently get when you call uppy.addFile(myFileObj). Maybe that’s ok? Not sure. My best bet right now would be to come up with a cool Redux-adapter, something close to what @richardwillars is proposing.

That said, we are open to more detailed proposals and PRs, of course. I would like this to be easy to use with React/Redux, as well as without.

is there a way to import only the core module and the plugins being used

Yes, sure! This should work and only bundle core and 2 plugins:

// instead of import you could: const Uppy = require('uppy/lib/core')
import Uppy from 'uppy/lib/core'
import DragDrop from 'uppy/lib/plugins/DragDrop'
import Tus10 from 'uppy/lib/plugins/Tus10'

const uppy = Uppy()
uppy
  .use(DragDrop, {target: 'body'})
  .use(Tus10, {endpoint: '//master.tus.io/files/'})
  .run()

[arturi]

I would like to summon @goto-bus-stop here too :)

[richardwillars]

@arturi Done! https://github.com/transloadit/uppy/pull/216

[goto-bus-stop]

I don't have a whole lot to add re: state. Mirroring to redux like that sounds like a :100: idea for redux apps!

The core module instead of keeping state could expose the events, and utilities for the developer to use?

I think having a decent programmatic API to Uppy's state should address most of the use cases. If it's the developer's responsibility to keep track of files and file progress etc, this can get quite complex. We have some API methods on the Core class already, I'm not sure if it covers everything someone might want to do at the moment, but it does the basics like adding/removing files and starting uploads. With the API eg. a Redux middleware could be written that turns actions dispatched from your app into Uppy API calls. That, paired with a plugin like @richardwillars's, would allow controlling Uppy with Redux actions and rendering a React component UI based on the Uppy state, with Uppy's progress updates and everything.

[oyeanuj]

@goto-bus-stop @arturi Thank you for explaining your point of view here, its very helpful. I want to think more about different paradigm/solutions (and I know @hedgerh has opinions on this) but I wanted to highlight one thing that @arturi mentioned:

And then Uppy is more like a glue that ties uploaders and UI together, normalizes things like progress in files, updates from encoding backends and things like that.

This is my imagination of the most flexible scenario where Uppy does the hard part, and connects everything but nothing more. For example, having a consistent shape for the file object would then be as simple as calling Uppy.newFileObject(userDroppedFile).

I am sure I am overlooking complexity inherent in file upload but the visualization of Uppy as a stateless glue sounded intuitive to me.

[arturi]

For example, having a consistent shape for the file object would then be as simple as calling Uppy.newFileObject(userDroppedFile).

That sounds like a good idea, we could expose something like that. Then you manually pass those files to uploaders and subscribe to their events. Sounds like possibly a lot of repetition.

[hedgerh]

@richardwillars approach is a start, but it's going to cause issues for people doing time travel debugging in redux dev tools. It's also going to cause issues if someone needs to re-hydrate their state, for instance:

• If my user closes the browser tab, I want to return them to their previous state when they come back.
• I want to record all actions a user performs in my app. If there's a production error, I want to send off a log of the current state and all actions performed, so I can replay every action to recreate exactly what went wrong.

In all of these kinds of use cases, the Uppy internal state is going to be de-synced from the Redux state.

[arturi]

Valid points! However:

1. Some of this may not work well with local files, because you can’t store/restore them if you navigate away from the tab, due to browser security.

2. Developers can’t expect all third-party libraries to not have some internal state. Whether it’s an image slider, Stripe payment widget, Intercom support chat, filestack-react or something else, they are going to have some internal state, which is not stored in your app, and work their magic. And then you communicate with that widget/plugin via an API and subscribe to their events. At least that’s my observation.

Maybe providing more flexibility for devs could be Uppy’s advantage. But I’m still having a hard time wrapping my head around combining manually selecting files, adding them to Redux, then listening to progress events from uploaders and setting state in Redux, and all that, with the current approach. Maybe we could have a whole separate universe with plain JS and then also React/Redux components and actions. Maybe we could just expose more APIs, for all uploading plugins, for example, and allow using them directly, what @hedgerh was doing at some point. But something tells me providing proper adapters or Redux actions is a better way to go. What @goto-bus-stop said:

With the API eg. a Redux middleware could be written that turns actions dispatched from your app into Uppy API calls. That, paired with a plugin like @richardwillars's, would allow controlling Uppy with Redux actions and rendering a React component UI based on the Uppy state, with Uppy's progress updates and everything.

[richardwillars]

Some of this may not work well with local files, because you can’t store/restore them if you navigate away from the tab, due to browser security.

For the product I'm working on we need to have uploading continue even when the user jumps to another part of the website (outside of the react app) or refreshes the page. Actually managed to get this working as a proof of concept using service workers. It carries on uploading even when the tab is closed, then when the page is reloaded it'll repopulate all the Redux state with the current status etc. Works surprisingly well! Happy to create a gist if it's something that someone might find useful..

[kvz]

That sounds amazing, I would love to see such a thing @richardwillars! We'll then discuss with the team and anyone interested if/how to bake in support for this.

[kvz]

Some of this may not work well with local files, because you can’t store/restore them if you navigate away from the tab, due to browser security.

Would it be too crazy to store the blob in local storage in cases where the size if < x bytes and if we'd make them expire at the same time that tusd expires the uploads?

In all of these kinds of use cases, the Uppy internal state is going to be de-synced from the Redux state.

Sorry for repeating myself but would all of this not be solved if you'd proxy all state to an adapter you could plug in, meaning there would be a single source of truth/subscription? So that by default there's an implementation of an interface that keeps the state local in Uppy. But if the developer so chose to drop in an implementation of that same interface that connects their own store, Uppy state would 'simply' live there?

[richardwillars]

That sounds amazing, I would love to see such a thing @richardwillars! We'll then discuss with the team and anyone interested if/how to bake in support for this.

To avoid cluttering this thread I've created a new one at https://github.com/transloadit/uppy/issues/237

[arturi]

Sorry for repeating myself but would all of this not be solved if you'd proxy all state to an adapter you could plug in, meaning there would be a single source of truth/subscription? So that by default there's an implementation of an interface that keeps the state local in Uppy. But if the developer so chose to drop in an implementation of that same interface that connects their own store, Uppy state would 'simply' live there?

@kvz as Harry said, that works partly. So the state will be in sync, but actions won’t, and that’s like Redux’s selling point. ADD_FILE --> START_UPLOAD --> PROGRESS_UPDATE --> PROGRESS_COMPLETE — if you implemented Uppy-like thing in Redux yourself, you’d have these actions and would be able to go back step by step, cause all steps are like UPPY_STATE_UPDATE. That’s my understanding.

We could, however, improve this, if we follow Redux-like conventions and implement Redux time travel without Redux:

Or communicate with the extension directly:

Or switch to Redux, as we also discussed at some point, and expose that.

[arturi]

@richardwillars thanks for the Service Worker tip and the code! We’ll look into this. Having upload continue after a page refresh sounds cool! 😎

[kvz]

Thanks for clarifying Artur! That helps. I wonder if the there is a lightweight Redux alternative, like .. Predux ? 😱🤗

Sent from mobile, pardon the brevity.

On 28 Jun 2017, at 18:14, Artur Paikin notifications@github.com wrote:

@richardwillars thanks for the Service Worker tip and the code! We’ll look into this. Having upload continue after a page refresh sounds cool! 😎

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[arturi]

There are, but Redux is relatively small, 4KB. But if we just use it internally, nothing changes, we need to expose actions and find a way to sync with the Redux used in the app Uppy is imported to. Correct me if I’m wrong someone please :)

[kvz]

Ah ok redux already is lightweight :) sorry, I'm way out of my comfort zone here.

But to me it sounds like going redux might solve a few problems, especially if people are able to inject their own redux so that all state lives in their app.

Again, likely not seeing the entire picture here, and also not the downsides. I guess one is that non-redux state management might still require hacks, and ofc the heart surgery in uppy to get there

Sent from mobile, pardon the brevity.

On 28 Jun 2017, at 19:22, Artur Paikin notifications@github.com wrote:

There are, but Redux is relatively small, 2-3 KB. But if we just use it internally, nothing changes, we need to expose actions and find a way to sync with the Redux used in the app Uppy is imported to. Correct me if I’m wrong someone please :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[oyeanuj]

If the team chooses to go with redux for internal state-management, I feel like having the following three levels of abstraction would make it really flexible -

#names just illustrate concept, not actual suggestions

UIWrapper: This is what current Uppy UI would be - the solution that most would use.

  WrapperAroundTheCore: Has the state, lets say, redux, but would allow others to add other kind of wrappers. But essentially this wrapper, manages the state.

    TheCore: Has no state, only has functions exposed which do the heavy lifting.

This way, based on user's needs, they can pick whatever level works best for them. I might pick TheCore or WrapperAroundTheCore whereas someone else might choose UIWrapper. Essentially, Uppy's default implementation would just be default wrappers that can be swapped by the user.

[richardwillars]

Moving off the topic of Redux for a minute, I wonder if it might be worth splitting Uppy into several different repositories / npm modules. Uppy could literally be the core module (and plugins), exposed via an API.

You could then have uppy-react, uppy-angular, uppy-vue etc, which are literally the UI elements and they just have Uppy as a dependency and talk to it via the API.

This feels a lot more modular and it's going to be a lot easier to maintain the project, keep extra dependencies out of the project, stops it getting bloated, and every time a new JavaScript rendering framework comes up we won't have to update the core Uppy repository.

You could even do the same with all the plugins (some plugins are view rendering specific, so breaking them into their own repos means you could have uppy-plugin-dragdrop-react, uppy-plugin-dragdrop-angular etc).

Any thoughts?

[goto-bus-stop]

@richardwillars We want to do that when things get a bit more stable, but for now it's nice to have everything in one package—no need to worry about syncing versions, if a patch needs to change something in core and also in plugins it's just one PR, and there's one place for issues. Maybe we'll move to a Lerna monorepo when we get closer to a 1.0 release, and split it up in different npm packages then, so using uppy as a dependency won't include a whole ton of stuff you don't need.