microsoft / FluidFramework

Library for building distributed, real-time collaborative web applications
https://fluidframework.com
MIT License
4.72k stars 532 forks source link

0.35 CRA-Demo Script and Feedback #5253

Closed micahgodbolt closed 3 years ago

micahgodbolt commented 3 years ago

@fluid-example/cra-demo 0.35 release

Follow the tutorial then leave comments below

We'll repeat this process with future releases

* This demo is a work in progress. There will be rough sections that need refactoring or refinement

micahgodbolt commented 3 years ago

This demo is out of date. Please use the current demo found at aka.ms/fluid-cra


Demo introduction

In this demo you will be doing the following:

  1. Install Create-React-App with Typescript
  2. Install Fluid and Fluid Data Objects
  3. Import KVpair and Fluid Static
  4. Update the view
  5. Start a custom hook
  6. Loading the KVPair data object
  7. Syncing our app state with Fluid data
  8. Run the app!

1. Use Create-React-App with Typescript

npx create-react-app my-app-name --use-npm --template typescript
cd my-app-name

2. Install Fluid and Fluid Data Objects

npm install @fluid-experimental/fluid-static @fluid-experimental/data-objects

* These are still experimental packages, and not ready for production

Lastly, open up the App.tsx file, as that will be the only file we need to edit.

3. Import KVpair and Fluid Static

Fluid gives you access to methods to boostrap a new Fluid container and attach DataObjects to it.

KeyValueDataObject will provide you with a fully scaffolded distributed data structure to store "key value pair" data and subscribe to change events. The KeyValueInstantiationFactory is required by Fluid to instantiate the KeyValueDataObject.

// App.tsx
import { Fluid } from "@fluid-experimental/fluid-static";
import { KeyValueDataObject, KeyValueInstantiationFactory } from "@fluid-experimental/data-objects";

3.a Add the getContainerId function

The Fluid class helps you create or load a Fluid container. As you build your application, you'll eventually track these containers yourself. For now, getContainerId function either loads the container identified by the hash in the URL or creates a new container for you.

This is an area we'd like to improve, but, for now, paste this code below your imports.

// below imports
const getContainerId = (): { containerId: string; isNew: boolean } => {
    let isNew = false;
    if (window.location.hash.length === 0) {
        isNew = true;
        window.location.hash = Date.now().toString();
    }
    const containerId = window.location.hash.substring(1);
    return { containerId, isNew };
};

4. Update the view

In this simple multi-user app, we are going to build a button that, when pressed, shows the current time stamp. This allows co-authors to see the most recent timestamp at which any author pressed the button.

To start, remove all of the existing Create-React-App returned markup and replace it as shown below.

You can see that this UI requires a data object and setPair functions to work, so we'll add those above and pull them out of a function we need to write, called useKVPair. The plan is for data to be a simple JavaScript object, where setPair sets a key value pair on that object. This allows us to write out data.time once the value is set by the button click.

function App() {
  const [ data, setPair ] = useKVPair();

  if (!setPair) return <div />;

  return (
    <div className="App">
      <button onClick={() => setPair("time", Date.now().toString())}>
        click
      </button>
      <span>{data.time}</span>
    </div>
  );
}

5. Start a custom hook

Working in React, one of the best ways to abstract complex, reusable functionality is via a custom hook. Custom hooks are functions that have access to the built in React hooks like useState and useEffect which we'll need in order to load our Fluid DataObject, and track our local state.

Hooks are just functions with stateful return values. So our hook will return data of type KVData, and a method of type SetKVPair which will pulled in async.

These two returns are all that we'll need to build out our sample app. In more complex scenarios you might use a reducer pattern to pass down a set of dispatchable actions, rather than giving direct access to the SetKVPair.

// above function App()
type KVData = { [key: string]: any };
type SetKVPair = (key: string, value: any) => void;

function useKVPair(): [KVData, SetKVPair | undefined] {
    return [data, setPair]
};

6. Loading the KVPair data object

The first part of our hook will load the KVPair Data Object into a place where we can use all of its built in functionality. With the KVPair we'll be able to set data onto the Fluid data structure, listen for changes via the on method, and update our local app state anytime those changes occur. With a few lines of code we'll have a UI that reads, writes and reacts to incoming changes from this multi user application.

6.a Create a place to store our dataObject

Since we're working with async functions, we will need a place to store our KeyValueDataObject once it is loaded. This is why we're using React hooks, because inside of a hook, we can use React's useState to create a stateful value and a method to modify that state.

// inside useKVPair
const [dataObject, setDataObject] = React.useState<KeyValueDataObject>();

6.b Create/Load the Fluid document and data object

Now that we have a setter method, we need to make sure that our create/get flow runs just once, on app load. Here we'll use React's useEffect because it allows code to be ran as soon as the component loads, and re-run only when specific values change, which by setting the dependency array to [], means never.

The bulk of this useEffect hooks is the async load function that starts by getting or creating the fluidDocument. The FluidDocument class then allows use to get or create one or more data objects. This could be multiple KVPairs, or other DataObjects that you define.

// inside useKVPair
React.useEffect(() => {
    const { containerId, isNew } = getContainerId();

    const load = async () => {
        const fluidDocument = isNew
            ? await Fluid.createDocument(containerId, [KeyValueInstantiationFactory.registryEntry])
            : await Fluid.getDocument(containerId, [KeyValueInstantiationFactory.registryEntry]);

        const keyValueDataObject: KeyValueDataObject = isNew
            ? await fluidDocument.createDataObject(KeyValueInstantiationFactory.type, 'kvpairId')
            : await fluidDocument.getDataObject('kvpairId');

        setDataObject(keyValueDataObject);
    }

    load();

}, [])

Once the DataObject is returned we assign it to our dataObject state variable, and now we have access to all of the KVPair's methods, including set which we pass down as the setPair function by adding const setPair = dataObject?.set;

Here's our hook so far.

function useKVPair(): [KVData, SetKVPair | undefined] {
    const [dataObject, setDataObject] = React.useState<KeyValueDataObject>();

    React.useEffect(() => {
        const { containerId, isNew } = getContainerId();

        const load = async () => {...
        }

        load();

    }, [])

    const setPair = dataObject?.set;

    return [data, setPair];
}

7. Syncing our app state with Fluid data

It is possible to avoid syncing data between Fluid and app state, but for this demo we will have React state drive our UI updates, and sync Fluid data into our React state any time that Fluid data changes. The advantages of this approach are:

  1. We leverage React's ability to update its UI based on changing state (vs forcing a re-render).
  2. In the real world, React state will often be a subset of the entire Fluid data.
  3. An MVC/MVP approach will require Fluid data (as the app database) to be translated into queries passed into views anyway.

7.a Create a place to store our KVPair data

Just like in our dataObject example in step 6, we are going to use React's useState to store the data we sync in from Fluid. In our case, we're going to dump the entire store into state, but in real life examples this syncing would be selective based on if data pertinent to this view had changed.

This state is shaped like a normal JavaScript object (great for view frameworks), and we'll start with an empty default value so that we don't need to worry about an undefined state.

// inside useKVPair
const [data, setData] = React.useState<{ [key: string]: any }>({});

7.b Listen for changes and sync data

Setting up listeners is a common usecase for useEffect, and this is exactly what we're going to do. The main difference between this example and the one above (6.b) is that on first render we won't have access to the dataObject, and will need to wait for it to load. So we only set up our listener if the dataObject is defined, and we'll make sure the useEffect is fired any time that the dataObject changes (i.e. after it changes from undefined to defined).

// inside useKVPair
React.useEffect(() => {
    if (dataObject) {
        const updateData = () => setData(dataObject.query());
        updateData();
        dataObject.on("changed", updateData);
        return () => { dataObject.off("change", updateData) }
    }
}, [dataObject]);

The function we want called on load, and any time that Fluid data changes will be called updateData. This function syncs our React state, via setData, to our Fluid data. Here we can use the dataObject's query() method to return an object with all of the key value pairs stored in Fluid.

Lastly we return the off method to remove the listener as soon as this React view is removed.

8. A working application

To see this application working we first need to fire up a local Fluid server called Tinylicious

npx tinylicious

Then we're ready to start our React app

npm run start

They both happen to use port 3000, so CRA will ask if you want to use 3001. Just hit enter

When the app loads it will update the URL. Copy that new URL into a second browser and note that if you click the button in one browser, the other browser updates as well.

DanWahlin commented 3 years ago

Nice and easy to go through. A few things to mention on the walk through:

  1. I like the approach. It'll be familiar and comfortable to React developers (at least those using hooks).
  2. Abstracting things out into the custom useKVPair() hook works well and should make it easier to port this to other front-end frameworks as well.
  3. I might be helpful right after they get the project going and install the dependencies to give a high-level overview of what they'll be doing so that when they perform each step they already have an idea in their mind of what they'll do. A visual that breaks things out into high-level objects might be helpful as well. This is super quick and off the top of my head, but something like this for the high-level overview:
    • Create a React app
    • Add a custom useKVPair() effect that does the following:
    • Handles loading a Fluid document and KeyValueDataObject used for real-time collaboration.
    • Provides a way to set data on the KeyValueDataObject and subscribe to changes made by collaborators.
    • Update the App component's JSX to add a button and display date information.
    • Start a Fluid server called Tinylicious.
    • Start the React app.
  4. It'd be helpful to add some basic comments (minimal) on key lines in the demo to explain what is happening. Pretty easy to follow for those who know something about Fluid but would be challenging for those new. I know it's early, but wanted to mention it. :-)
  5. The more you can have people add code in the order it appears the better. For example, const [data, setData] = React.useState<{ [key: string]: any }>({}); isn't added until later so for awhile I thought I missed adding something in the hook and went back to look. I'd have them add the shell for useKVPair() and then slowly add the code in the order it appears so that everything is there as they add things (explaining along the way like you do now). Maybe start with this and explain the key building blocks first:
function useKVPair(): any {
    // useState() here

    React.useEffect(() => {

    }, [])

    React.useEffect(() => {

    }, [dataObject]);

    // return object here

};

Then have them add the useState() calls:

const [dataObject, setDataObject] = React.useState<KeyValueDataObject>();
const [data, setData] = React.useState<{ [key: string]: any }>({});

Then have them fill in each useEffect(). And finally have them add the return value. By doing it that way they start really small, understand the basic building blocks, and then add to them over time. Just my two cents there.

  1. While I'm a big fan of custom types in "real" apps, for demos I think they tend to convolute things. For example, instead of having:
function useKVPair(): [KVData, SetKVPair | undefined] {
  ...
}

It might be better to just use any initially. I know....evil, but you could remove the custom type definitions and simplify the return signature initially which makes it way less intimidating.

Using more custom types is one of the things I've found that discourages people who may be new to those concepts. Especially those who come from a pure JavaScript background and haven't used TypeScript much. They tend to feel intimidated right upfront which ends up affecting everything they do in the walk through.

After they get it going you could add an optional step at the end to enhance the return signature of useKVPair(). Doing that lowers the barrier to entry and you could always mention that the any return type is there to keep it simple and that there will be an option at the end to enhance it with a custom type.

Comments on the code:

  1. I feel like there's an opportunity to cleanup the load() function potentially by having something like a createOrGetDocument() that takes isNew as a param. It just feels like duplication having createDocument() and getDocument() right next to each other even though they do different things of course. Same for getting the data object. Not a huge deal of course, but something that could really simplify things while still allowing people to use the create and get functions directly if they want.
  2. Not a fan of [KeyValueInstantiationFactory.registryEntry] but it sounded like that may change from one of our calls.

To wrap up, really nice job on this so don't let my comments above infer otherwise. They're just suggestions that I think can potentially take it to the "next level". I really like the direction this is going.

reinvanleirsberghe commented 3 years ago

Managed to get it work without hassle, so pretty clear for me...

Would like to see the same example using the @fluidframework/react though. Just to see what's the difference...

Also is the KVPair suitable to sync/share large objects or should we use another DDS for that? That's something that is not always clear for me. Which DDS's you should use for different kind of datasets.

micahgodbolt commented 3 years ago

@reinvanleirsberghe Thanks for the feedback! A few quick replies:

KVPair is pretty unrefined at the moment. It's meant to be a quick, simple DO you can pick up and use for those 80% of the cases where you simply need to sync a bit of data in an app

For larger data sets with more complicated APIs, I'd certainly recommend building your own purpose built DOs just like we did with KVPair. That process, and the challenge of picking the correct DDS for your type of data is beyond the scope of this demo, but certainly an were we hope to dig into in the future.

The goal of this demo is, as a react developer, how am I supposed to interact with ANY DataObject, and how can we make that experience the best it can be.

Thanks again for the feedback!

LarsKemmann commented 3 years ago

Here's my feedback, by section, intentionally without having read any of the other comments or responses yet. (@SamBroner @ahmedbisht FYI.)

1)

2)

3)

3.a)

4)

5)

6)

6.a)

6.b)

7)

7.a)

7.b)

8)

Summary

2/5 stars, for at least being better than previous Fluid docs but still leaving me wishing I'd spent my Saturday differently. This even though I love the promise of Fluid and consider myself an expert-level developer/architect. I've actually built several real-time collaboration systems from scratch and did the equivalent of master's thesis research/work on the subject in a startup for two years. I'm going to keep at it because I'm extremely motivated, but unless Fluid provides more of its capabilities in a format that's at least somewhat adoptable by 200-level developers I can pretty much guarantee you it's not going to take off. I was hoping to be able to start an actual Fluid app today for a real-world business problem at our company (which my wife graciously gave me 5 hours of our weekend to do) and all I have to show for it is a single shared key-value in ~75 lines of code. 😢

LarsKemmann commented 3 years ago

Now that I've read some of the other feedback/responses, I feel even more strongly about my 2/5 star review.

@micahgodbolt

The goal of this demo is, as a react developer, how am I supposed to interact with ANY DataObject, and how can we make that experience the best it can be.

I know that the changes I've asked for (like having a single @fluidframework/react NPM import with useFluidMap, useFluidTable, etc. hooks that are all I need to use those data structures) are not something that'll happen overnight, but I sincerely hope it gets prioritized before any kind of public preview/promotion (e.g. at Build). An API that requires as much boilerplate code as this one just to use a single shared time property is not adoption-ready.

I think part of the problem is that most of the Fluid examples I've come across, including this one, only ever show a single property, rather than an actual app. Maybe I'm just missing something and the APIs actually start to compose really well once you scale up from this point, but I don't see how that's possible.

Consider Redux, widely regarded as one of the most complex things you can do in React but also very widely adopted. The number of lines of code to create a shared model, reducers, etc. is 20 or less, and most of that is the user's data -- not using/composing framework APIs that the user has to learn. Once that is in place, it's ~5-10 lines of code to add a new property, reducer, etc.

Fluid is clocking in at 60+ LOC for a single shared value, most of which is highly intricate API composition that requires extreme attention to detail (there are some very nuanced things to consider in the hooks code in the sample above, as you call out in the supporting explanations), and there's no clear way to see that the next data object/type I want to add to my app won't also require that same level of effort since none of that code appears to be reusable in its current form.

skylerjokiel commented 3 years ago

@LarsKemmann , thank you for taking the time to both go through our tutorial, and write such a thorough critique. We both need it and appreciate it greatly. Feedback is critical to our progress, and hopefully with iteration we can eventually arrive at least at a 4/5 star review. 😊

For context, this tutorial is a first step in a larger effort and hopefully I can provide a bit more framing around our thinking.

The Fluid Framework has a ton of raw power and is designed to be a base layer for building complex real-time applications. This power comes at the cost of complexity through abstractions. When developing the first versions of the framework one of our core principals was to provide the users (usually ourselves) the option to do whatever they wanted, wherever they wanted. This produced the foundation of the modern core framework and enables a lot of the complex (1000 level) scenarios we have within Microsoft. But abstraction comes at the price simplicity. When you can do everything you can't easily do anything.

We have historically been in the mindset that developers will build within some form of the Fluid ecosystem. There is a lot of benefits you get from building within the framework that we really want to bring to developers. This, "in the framework model," is apparent in our current HelloWorld experience that involves the developer building then consuming their DataObject. We've found, as you have, this is way to heavy for developers getting started. Lots of concepts to solve a simple problem like a dice roller. To make it easier for developer we are focusing on providing pre-built DataObjects that developers only have to consume. What you are seeing here is the initial attempt at providing this model of DataObject consumption.

This consumption model is a focused on the Fluid Framework being a Data+API layer in your webstack toolkit, and we have three primary principals that guide our decision making.

  1. Simple - Make it easy to use and simple to get started
  2. Embrace the web - Fluid needs to work with existing web technologies
  3. Depth is available - Provided clear paths to additional functionality

We are working to strike the balance is between simplicity for 100-200 level developers and depth for 300+ level developers. As you've pointed out we are still missing the mark on both and this is where community feedback and iteration is so critical. Your comments around reducing the number of concepts and specifically removing factory concepts is on our radar and something we will be addressing. Your comments around spending 5 hours and ending up with a single shared key-value in particular makes me very sad. We have support for alternate DataObjects and DataObject composition on our radar but your comment brings up other thoughts. We need to do a better job setting expectations for these types of engagements and specifically what we expect a developer to be able to do with the end product. This is definitely a simple 100 level learning tutorial with no path higher application and it should be represented that way.

The other difficult problem we are trying to solve for is scenario story telling from 100 to 200 to 300+ level developers. No developer will build an entire website with a collaborative dice roller but as begin to consider real applications we believe strongly that developers need to be able to bring web-technologies that work best for them. The tradeoffs here are similar to paragraph above but with a focus on ensuring someone who writes a 100 level application doesn't need to re-write their code to take advantage of depth within the framework. Considering the breadth of features the framework has, and the different points of integration, keeping the right amount of depth available to developers is a challenge.

Finally I want to talk about view frameworks. The idea of a @fluidframework/react package is something we've considered on multiple occasions. We actually have an existing package already that does this but has not been worked on in a while. As I mentioned above we've taken a strong focus on Fluid being the Data+API layer of the webstack. This is intentional because as a very small team working on developer experiences we believe that in order to make Fluid world class we need to first focus on the fundamentals. But, we also recognize that data in itself is not very exciting. We need to empower developers to build applications which means we need to embrace web technologies and integrate well with view frameworks. Kind of a chicken and egg problem we have here. The strong feedback I'm getting here is a need to understand the pieces and what is within your control as a developer. As an professional developer you should be able to walk away from this and say "I get how I could integrate this into my application." Instead you are asking why we even expose it. We cannot teach every scenario and we need to get to a state where developers can learn concepts that they can apply to their unique scenarios.

Finally, you've included a lot of pointed actionable feedback around both technical improvements and storytelling. I'll make sure these are included in our planning. If you feel particularly passionate about any issues please make sure your log an Issue.

Thanks again for taking time on your weekend. I've learned a lot and look forward to future engagements! 😊

flvyu commented 3 years ago

@micahgodbolt Thanks for putting this tutorial together. I followed the tutorial and was able to get the app working. I think this is a great step towards making Fluid much easier to use and integrate in React apps.

Where the steps clear? The steps were mostly clear.

I do think more information could be provided on why we were using certain Fluid APIs, methods, enums etc.

Overall, I think it is more important to focus on the Fluid API itself vs how the React APIs work.

For example, it would be great to add information on what createDocument does and what a document is modeling and abstracting. Without an explanation, this can lead to other questions like:

  1. Why do I need a document?
  2. What is the difference between a document and a container?
  3. What are the relationships between document and container?
  4. ...other stuff related to Fluid

Impression on Approach: I really like how you used hooks to implement the logic for the data objects. This makes it much easier to add data sync logic in our apps and hopefully reduce the amount of code needed to get things working. The more we can abstract into hooks, the easier it will be to work with and use.

Other thoughts

  1. "Install Fluid..": I'm not sure what install Fluid means since I see we install the Fluid Static package. It might be clearer to explain what the Fluid Static Package is.
  2. In the end, it would great if we had a the final implementation of useKVPair in one place. This will allow us to double check we have the correct code by looking in one place instead of going through multiple sections. I did miss a line of code and had an error in my app. Similar issue that @LarsKemmann pointed out in step 7.a
  3. Since we use useEffect twice, it might be cleaner to add the import for it to the top.
  4. Does this have to be implemented in Typescript? It's only a little bit of TS, but new developers might be more familiar with regular JS.
  5. For me, I think Section 7 does a great job explaining the benefits of using React to sync the Fluid data.

For the point of this tutorial, I think the app we built makes sense, but to make Fluid more appealing I think a more interesting example would be more fun :)

I'm looking forward to using the experimental packages soon!

reinvanleirsberghe commented 3 years ago

Fluid.createDocument and Fluid.getDocument do not exists anymore as from version 0.36.0. Updating the steps should be helpful :)

micahgodbolt commented 3 years ago

@reinvanleirsberghe thanks for the reminder. Several things actually changed and my plan was to close this issue now that the 0.36 demo is out.