search

dj-stripe / dj-stripe

dj-stripe automatically syncs your Stripe Data to your local database as pre-implemented Django Models allowing you to use the Django ORM, in your code, to work with the data making it easier and faster.
https://dj-stripe.dev
MIT License
1.55k stars 472 forks source link

Docs are broken in several ways/places #1841

Open jacklinke opened 1 year ago

jacklinke commented 1 year ago

What did you run into?

Trying to read the documentation to better understand this project, but much of the information is missing from pages or incorrectly formatted/garbled.

Why would it be helpful to document or fix this?

To which section of the docs does this belong?

It looks like much of these issues crept in during #1451

jacklinke commented 1 year ago

More specifically, the pyproject.toml references Homepage which is broken, and Documentation, which is not broken.

Is there a reason for two different locations for the same project documentation, using different templates and settings?

Clicking on "Documentation" from the project Homepage takes you to https://dj-stripe.dev/installation/ - the equivalent of https://dj-stripe.dev/dj-stripe/2.7/installation/

Also, searching Google for this project takes you to https://dj-stripe.dev/ - the version of the site with broken/missing docs.

kavdev commented 1 year ago

@jacklinke yep, docs are definitely a pain point in dj-stripe right now. Thank you for taking the time to write this up. Any contributions are appreciated; PRs are welcomed.

jacklinke commented 1 year ago

@kavdev I guess the first thing would be to determine the desired end-state.

Current state

https://github.com/dj-stripe/dj-stripe/tree/master/docs

https://github.com/dj-stripe/dj-stripe.github.io/tree/master/docs

Recommendation

Of the two versions of docs, the version in dj-stripe.github.io repo is far more visually appealing and maps more nicely (no extra directory after the domain), but because it is in a separate repo it is currently unable to take advantage of the modifications/hacks @arnav13081994 used to simplify and DRY the docs in the main dj-stripe repo.

I propose porting the plugins used with mkdocs in this repository to the dj-stripe.github.io repo, modifying the github actions doc build flow to git clone this repo's contents into a temp folder that can then be used to pull out the relevant docstrings, and making dj-stripe.github.io and https://dj-stripe.dev the canonical documentation home for dj-stripe.

Does this seem sensible?

kavdev commented 1 year ago

Yep, sounds like a solid plan. I would be very intrigued to see a PR with a POC with one section working.

For some backstory, we had a team create the website and port our docs over. It got close, but we're still in a bit of limbo.

I would love to see our docs fully contained within dj-stripe.dev. That would enable us to drop rtfd and make life a bit less confusing for our users.

I recently spun up another project with dj-stripe and got lost for a sec looking for some of the docs I wrote 🙃. This would be a good step in the right direction.

cc @jleclanche for your thoughts before I give this a green light. I could be out of the loop on recent dev plans.

jleclanche commented 1 year ago

You're exactly right, and yes I'd like to see our docs properly generated into the website.

@jacklinke your recommendation is spot on. Is this something you're willing to help with? I know @kavdev tends to be too busy to contribute to dj-stripe these days, and I myself only have time on specific occasions.

jacklinke commented 1 year ago

My next 2 weeks are pretty much 100% spoken for, but I would be happy to get to work on a PR starting 1 December. If that works for y'all, then I'll add it to my calendar and update here at that point as things progress toward the PR :smile:

kavdev commented 1 year ago

@jacklinke sounds good! Happy to review whatever comes through

jleclanche commented 1 year ago

Likewise, help is super appreciated!

onekiloparsec commented 1 year ago

Hello guys. As far as I can tell, @jacklinke had no time to submit a PR for that issue, right? Hence my modest suggestion in this PR

jnns commented 10 months ago

The docs are still in a bad shape currently. As far as I understand the situation is as follows:

It seems that dj-stripe.dev is advertised as the canonical place although it is broken and there are still plenty of references to dj-stripe.github.io/dj-stripe.

Seeing that several attempts to fix the docs have been made already. I suggest to keep things simple and go back to using Readthedocs only. The repository dj-stripe/dj-stripe.github.io could then be removed and the domain dj-stripe.dev could link to the docs at readthedocs.

With the current state I don't want to invest time in fixing the content of the docs because I can't be sure that it will be found by the reader when they are looking for it.

onekiloparsec commented 10 months ago

Thanks @jnns for contributing to that important question. I do have the exact same feeling: I am afraid of investing time on the content of the doc because I am not sure it will end up in the right place. And we definitely need to improve (by a lot) the docs of dj-stripe.

My PR was an attempt to solve this, but is based on a tricky machinery, which failed. At that stage, I would agree with anything simple and straightforward.

@jleclanche @kavdev we need your help here (and a quick decision). Thanks in advance!

czue commented 7 months ago

Alright after reading through https://github.com/dj-stripe/dj-stripe/issues/1841#issuecomment-1607172588 and assuming that is the approximate path we want to take forward, I'd propose a few quick updates:

  1. Decide which is the canonical docs moving forwards. It seems like this should be the one that is automatically deployed (right?) so would be https://dj-stripe.github.io/dj-stripe which seems to redirect to: https://dj-stripe.dev/dj-stripe/
  2. Update all deprecated docs at the DNS level to redirect to that site (this would include https://docs.dj-stripe.dev/, and maybe others)
  3. Update the "documentation" navigation link in the header of https://dj-stripe.dev/ to point there (assuming that site is going to live on as a more polished marketing-style site for the project - I don't have my head wrapped around what it is and how it is different from the "pure" docs)

Since the docs build is broken, that will still point everyone to docs that stop at 2.7, but at least there won't be many different places to look for information and will set us up from an SEO and comprehensibility perspective to get everything fixed as soon as that last piece is put in place.

Maybe? I'm very new to the project so may be missing things...

(who controls the DNS for dj-stripe.dev? That seems a critical first piece of the puzzle)

jleclanche commented 7 months ago

@czue Thank you for looking into it!

  1. The canonical docs website should be on dj-stripe.dev. We should absolutely auto-deploy it with github actions.
  2. For DNS updates, just let me know what needs updating and I'm on it. Alex and I have cloudflare access.
  3. I don't think there's a difference really. The docs on the dev site are just using a custom theme for our docs generator and we can keep doing this.
jleclanche commented 7 months ago

PS - don't hesitate to @ me or even email me directly if you need urgent help. I'm in the middle of a fundraise so sometimes I miss things.

czue commented 7 months ago

@jleclanche alright, thanks for the heads up. How/where is dj-stripe.dev hosted and how is it currently intended to be built?

jleclanche commented 7 months ago

I believe it's hosted on github pages. This repo builds it: https://github.com/dj-stripe/dj-stripe.github.io

jleclanche commented 7 months ago

Note that there's two branches.

onekiloparsec commented 7 months ago

@czue Thanks for brining this subject up again. The main difficulty is that there are two separated repos: the main one with the code (and with the documentation that could be generated from it – easier for everyone I guess), and the one (https://github.com/dj-stripe/dj-stripe.github.io) which publishes the docs to the GitHub pages (and where the DNS points to).

For what is worth, here are my 3 PRs on the second/"publishing" repo, attempting to solve this issue: 2, 3, 4.

jleclanche commented 7 months ago

@onekiloparsec I am unconvinced that it's easier to have the docs in the same repo. The workflow in the separate repo just clones this one here; and we don't need to regenerate the doc every commit, we can just do it once per release.

Managing documentation-related PRs etc is easier in a separate repo IMO. But if someone takes up this project, I will take the help regardless how you want to do it :)

onekiloparsec commented 7 months ago

We agree. :-) I am not saying it's easier to have it in the same repo. Just that having it in a separate one requires to setup a little machinery.

czue commented 7 months ago

Thanks for the inputs both. I tried to get up and running building the other repo by following what the CI was doing, and managed to do that. I documented my steps here: https://github.com/dj-stripe/dj-stripe.github.io/pull/5

However, when I did this, the generated documentation site looked like https://dj-stripe.dev/dj-stripe/master/ and not like https://dj-stripe.dev/ (all the fancy styling was lost).

@jleclanche I'm pretty confused by this statement:

The docs on the dev site are just using a custom theme for our docs generator and we can keep doing this.

When I search the repo for, e.g. "Stripe Made Easy for Django Developers" it comes up in HTML files, but not anywhere that looks like it could be built. How was this initial website created? It doesn't seem like the above is plausibly true, unless I'm missing something.

Assuming that it is at least somewhat close to true - where can I find that theme and how do i build the docs using it?

jnns commented 7 months ago

I believe it's hosted on github pages. This repo builds it: https://github.com/dj-stripe/dj-stripe.github.io Note that there's two branches.

It looks like both branches contain just the generated static site (there are only .html files, no .md files).

I am unconvinced that it's easier to have the docs in the same repo. The workflow in the separate repo just clones this one here;

I could understand that docs are cloned into and published via dj-stripe/dj-stripe.github.io if it wasn't for the extra dj-stripe.dev domain: after all, https://dj-stripe.github.io is nicer than https://dj-stripe.github.io/dj-stripe. But when there's dj-stripe.dev anyway, what merit does the the intermediate repository dj-stripe.github.io have and doesn't it make the whole setup more complicated?

and we don't need to regenerate the doc every commit

It seems to be a simpler solution to filter by branch/tag (or changes in the ./docs/ folder) in the GitHub Action that builds the docs, not to create a separate repository with a questionably complex ci.yml.

jleclanche commented 7 months ago

@jnns I believe dj-stripe.dev is just configured to be dj-stripe.github.io. It's just a prettier domain name than the github one (and more future-proof).

jnns commented 7 months ago

That's what I mean. If we have a custom domain anyway, we can skip the additional repository github.com/dj-stripe/dj-stripe.github.io and have the custom domain point to the docs in this repo (dj-stripe/dj-stripe).

czue commented 7 months ago

It seems sensible to me that if it is true that the goal of dj-stripe.dev (the domain) is to host an unmodified, possibly differently-styled, version of the docs, then having a separate repo for it is not necessary (and net-negative).

I'm still unclear if the first half of that statement is true though. @jleclanche did you see my comment above? Does anyone know how to build the docs so they look like the version on dj-stripe.dev? Or, alternatively, how the https://github.com/dj-stripe/dj-stripe.github.io repository was created? There is text on the landing page that does not appear to be anywhere in the docs.

jleclanche commented 7 months ago

@jnns oh i see what you mean ... sorry.

Okay so the dj-stripe.github.io repository was created by @apedrob as part of a short contract with Evolutio to create the dj-stripe website and hook up the documentation to it. Unfortunately I think when doing this, they duplicated the documentation and didn't really tidy everything up. So, this is why I'm open to just not looking too much at how things are set up now and just set them up the Right Way moving forward.

Here are the key elements:

  1. The website itself should not be part of the dj-stripe repository. It's a logically separate project, and PRs for it should not be mixed with PRs for dj-stripe itself.
  2. There should only be one source copy of the documentation.
  3. That source copy is best hosted as part of the dj-stripe repository, otherwise it will go stale too quickly.
  4. The documentation should be published on the dj-stripe.dev website. So I think when generating the website, it makes sense to pull the dj-stripe repository and generate documentation for it.
  5. We should be hosting documentation for the older versions of dj-stripe on the website as well. How to do it? I don't know; maybe pull a list of branches when building?
  6. I don't think it's necessary to have live documentation of dj-stripe's master branch on the website ... but I think it's useful to have master documentation available somewhere so that people can peruse their own changes to it as they file and merge their PRs. Lower priority, but overall it's important to make it easy for people to contribute documentation changes.
jleclanche commented 7 months ago

@czue did you check this branch? https://github.com/dj-stripe/dj-stripe.github.io/tree/gh-pages -- IIRC this is where the auto-builds were happening.

czue commented 7 months ago

@czue did you check this branch? https://github.com/dj-stripe/dj-stripe.github.io/tree/gh-pages -- IIRC this is where the auto-builds were happening.

@jleclanche the text "Stripe Made Easy for Django Developers" does not appear anywhere outside of an html file on that branch or on the other branch. I cannot find any evidence of where it is coming from. Is it right that the current state is that nobody who is still working on the project knows how that site is built? Would @apedrob be able to weigh in?

Responding to these other points:

The website itself should not be part of the dj-stripe repository. It's a logically separate project, and PRs for it should not be mixed with PRs for dj-stripe itself.

I get the philosophical reasoning behind this, but if "the website" truly is just a github action configuration I think you could make the case that it's a more practical set up to keep it with the docs.

There should only be one source copy of the documentation. That source copy is best hosted as part of the dj-stripe repository, otherwise it will go stale too quickly.

Hard agree with these.

The documentation should be published on the dj-stripe.dev website. So I think when generating the website, it makes sense to pull the dj-stripe repository and generate documentation for it.

Maybe, though to other people's points, it'd be much more useful if the docs existed and were up to date anywhere before figuring out how to ensure they are on that particular site in that particular style.

We should be hosting documentation for the older versions of dj-stripe on the website as well. How to do it? I don't know; maybe pull a list of branches when building?

If we keep the docs on readthedocs then this is handled by them, right? Would that not be a more sensible middle ground until someone has the capacity to figure out the workflow on dj-stripe.dev?

I don't think it's necessary to have live documentation of dj-stripe's master branch on the website ... but I think it's useful to have master documentation available somewhere so that people can peruse their own changes to it as they file and merge their PRs. Lower priority, but overall it's important to make it easy for people to contribute documentation changes.

Agree, this is not a requirement, but definitely a nice to have

jleclanche commented 7 months ago

Maybe, though to other people's points, it'd be much more useful if the docs existed and were up to date anywhere before figuring out how to ensure they are on that particular site in that particular style.

Yes, I'm with you on this. Happy to leave it to rtd to host the old docs for now.

duaneking commented 2 months ago

@onekiloparsec I am unconvinced that it's easier to have the docs in the same repo. The workflow in the separate repo just clones this one here; and we don't need to regenerate the doc every commit, we can just do it once per release.

Respectfully, That is bad design. We should be regenerating them for every release, yes, but they should also be done every checkin as part of the build so that people can check them as needed and file bugs in a central place. Keeping them in separate repos is just creating more work and making it harder to maintain and use, and that is slowly killing the project.

We should simply place the docs and the code in one repo, the docs in its own subdir building a small docs package, and have the github pages repo deploy a copy of the main repo docs to github pages as a deployment, not a separate project; Splitting the docs and the code is a bad idea.

That would centralize the data and allow the github pages repo to be a devops repo without needing to worry about conflicts between the two repos, allowing the data to be maintained easier, with lower effort, in a centralized way, resulting in easier to maintain code and docs.

This unneeded added complexity is death by a thousand cuts; the github pages repo should pull in the docs and code from the main repo to simply deploy a docs package that is created as part of the normal core repo build.

This would solve most of the issues and make maintenance easier, resulting in less defects.

It honestly feels like this project's been sabotaged with un-needed complexity; I'm tempted to say that it feels like there are people actively working against the project, but I have yet to find actual proof of that so I'm going to keep those opinions to myself. However, it seems logical that somebody would want this project to fail because its existence allows others to be competition. As a result, I think the project should be as simple and transparent as possible, and so a centralized single repo for docs and code that makes things easier, with the github pages repo effectively being just a deployment of a package released by the core main repo, seems reasonable.

onekiloparsec commented 2 months ago

@duaneking I agree, this is useless complexity, and it has been way too much time. Now we need s reliable solution. Indeed, there are interesting competitors, and the value of dj-stripe is not especially shining, mostly because of lack of up-to-date documentation and examples on key points.

No need to look for sabotage. Let's focus on solving the issue. Maybe, new official contributors would also help to have MRs being accepted in a quicker way.

duaneking commented 2 months ago

the value of dj-stripe is not especially shining, mostly because of lack of up-to-date documentation and examples on key points.

I've personally been spending way too long trying - and failing - to get a working implementation deployed because the docs are either wrong or actively lie, so I'm wasting a lot of time being lead to dead ends and false starts. This makes using DJ-stripe an active business risk due to its current lack of support, and that's being cited as a reason not to use it by multiple business partners who say it would be best to roll our own solution and not use dj-stripe.

No need to look for sabotage.

No matter the reason for the issues, The level of knowledge about the project needed to create these examples and fix these issues is simply too high of a bar for someone to randomly start and be able to contribute. This is further proven by the fact nobody has done it in over a year, as well as the fact that doing so requires a level of access that only core maintainers have, so the fact this issue is marked as "an easy first issue" would honesty be a Hilarious joke, if the bar to get the docs accurate was not so high and did not require levels of access that a first time contributor simple does not have. Either way, The lack of effort by the core team that has the access is the problem. It's blocking efforts by everybody else who wants to contribute, including myself.

Let's focus on solving the issue.

I'm willing to put in the work to get something deployed, and share fixes as needed to upstream.

jleclanche commented 2 months ago

@duaneking No sabotage here, if you want to improve the docs I will support you in any way I can. And I don't know if you noticed but we are actively trying to get the project to be highly simplified for the 3.0 release. In fact, thanks to Stripe, we have budget to get this documentation issue fixed so if you want to get paid to do it, there's an opportunity to do so.

Unfortunately, writing documentation is a time-consuming task, and I'm three months behind on my rent because of personal reasons which means I cannot dedicate a lot of work time to dj-stripe right this minute.

To your point on unneeded complexity - look, I don't care really, it was just my opinion but if someone wants to invest time in maintaining it in-repo or out-of-repo, I will take it either way. But it used to be in-repo and it was not kept up-to-date any better than it currently is, and it's not that one design point that's keeping this from happening.

And tbh, dj-stripe is a project that I worked pretty fucking tirelessly on to get to a simpler and more predictable state. It used to be a LOT more complex and full of cruft and today it's a hell of a lot more predictable, even if the documentation hasn't kept up with that. So honestly, I'll thank you to keep these random useless accusations to yourself, they're pretty much never welcome in any OSS project, not just this one. (Your profile says you "take no crap"; neither do I, FYI)

There's an action plan in a comment above. If you disagree with it just give a heads up on what you want to do. I will assist however possible, and if you want to get paid for the job I can also make sure it happens. Any more off-topic comments past this point will get deleted; if you want to talk governance, there's an open discussion forum.

duaneking commented 2 months ago

thanks to Stripe, we have budget

Ok so why wouldn't the core maintainers want to get paid to do this and ease the burden on new contributors who lack the context to do this easily?

I'm three months behind on my rent

I'm sorry about your situation. If I understand correctly, and with all due respect, you say you have budget from stripe that you are intentionally not accepting to work on your own project despite being a core maintainer, and if I understand correctly, you are unwilling to accept that money and get paid to work on your own project, so as a result of that, you are behind on your rent? I'm sure I don't have all the information, but It really does seem on topic to ask why having the core maintainer use that funding to fund documentation updates isn't in the best interest of the project. I would be happy to help push back on stripe if they are the reason for this limitation, to help you. I would much rather the core maintainer benefit and in this case, the core maintainer would have more knowledge and be able to do the work more effectively, So it seems a reasonable business case that the stripe funding would have better value being paid to you for doing the work, and I'm confused on why that is not the case.

dj-stripe is a project that I worked pretty fucking tirelessly on to get to a simpler and more predictable state

Thank you.

(Your profile says you "take no crap"; neither do I, FYI)

I honestly respect the candor and the vulnerability and am not offended in any way. I'm glad we can talk freely, thankful that you are open to doing so, and I think its safe to say that both of us want what is best for the community of users who use dj-stripe. Thanks also for putting some more context out, it explains a lot and allows me to go to other people and explain this not as a lack of care, but simply a stack ranked priority. I firmly believe that communication like this is important and allows us to simply resolve issues and move on quicker, So thanks for being collaborative on that.

As for how I can contribute today immediately, I'm at a loss. I would love to be able to assist in creating working examples, yet I need a working example to do that and every time I tell myself "Duane, you are going to dedicate time to dj-stripe today, we are going to go downstairs, turn on all our monitors, light a scented candle, put on some chill jams, and force yourself to work on this" I get distracted with other issues that are less painful, and it's unfortunate, but part of that frustration is the inability to quickly get started so that I can start iterating, due to these documentation issues. It feels like a recursive dependency, And that might actually call out the value of the documentation issues getting fixed.

I started filing defects when I first started looking into this, in the hope of helping map a working path, but after not getting a response I stopped because I felt like my efforts where not helping map a way forward, and I was worried it was unwanted due to the lack of response. If that's not the case, then I will take another look.

Thanks again for the radical candor.

jleclanche commented 2 months ago

@duaneking I understand the confusion, and no, this has nothing to do with Stripe themselves, they've been nothing but extremely helpful. Unfortunately because of burnout a year or so ago, I am left mostly unable to write code anymore. I can still do code review, design & architecture, management, and I operate my own startups on these skills; but writing code/examples/documentation is one of those things I physically cannot do (I get severe headaches, dizziness, and worse).

But yes, help is absolutely 100% wanted. If you are willing to invest time into dj-stripe (be it as an active user or a contributor) I am absolutely willing to dedicate time to help you personally. You can book time with me here: https://cal.com/jleclanche/quick-meeting

duaneking commented 2 months ago

You can book time with me here:

Done.

onekiloparsec commented 2 months ago

@jleclanche sorry to hear about your difficult situation. It's impossible for me to really contribute to the code or any serious work. But I'd be happy to provide a few basic examples of what I do myself with dj-stripe to start with, and help @duaneking breaking the recusrive dependency... Keep us posted.

jleclanche commented 2 months ago

Thank you both.

I believe one other helpful entry point would be further work on this project: https://github.com/dj-stripe/djstripe-example-project

I'll try to make myself more available soon.

onekiloparsec commented 4 weeks ago

Hi. Any news?

duaneking commented 4 weeks ago

Hi. Any news?

@jleclanche and I spoke but then he stopped emailing me and I don't know why. I have been waiting for a response from my last email for weeks with no update, so I'm assuming he does not want my help.

A few things:

  1. The docs need a full rewrite from zero.
  2. Somebody should ask Google to remove the bad versions of the docs from its index.
  3. I get stackoverflow answers to problems I'm still having that tell me to use code that has since been deleted from the project.
  4. Syncing data from stripe should never be a manage.py command. That is just an insane design choice to require this be run constantly. The API should be updating its database when it creates the customer, updating its stored cache of the data on every API call, but that's clearly not happening or I wouldn't be getting "does not exist' exceptions thrown after I create a customer and try to Attach it to a specific user so I can simply reuse the same customer on a 1 to 1 basis per user and provide Subscription management capabilities that are consistent. Not even that seems to work as expected in my tests.
  5. I firmly believe that my efforts to get this working have actively harmed my mental health, so I have been avoiding it for the past week or so to protect myself. My experience with this project has seriously made me consider abandoning using stripe for anything in the future.
jleclanche commented 4 weeks ago

Hi @duaneking, last contact I had with you was on March 5 in which you asked for tax advice; I followed up with an email on March 6, in which I suggested a way to move forward as well as gave some advice on how to improve your CV. Either you missed it or you ignored it, but that is the last contact I had with you with no follow-up since. I assumed you were no longer interested in helping, and I'm confused what harm has come to your mental health given that I've seen no activity since...

As to point 4, the manage.py command is not a production requirement, it is a helper to get things synced in case for some reason the db has fallen out of date (eg. after a database wipe, crash recovery, etc)

duaneking commented 4 weeks ago

  1. You seemed to have misunderstood the intention of that email. You literally offered me money to help you, but you didn't explain how that was going to work as I have no company set up and actively avoid income from other nations due to the tax liability. I was simply doing due diligence and asking follow up questions.

  2. I don't ignore messages to me. You can see how responsive I am here, for example. I actively go out of my way to be as responsive as possible. I'm also responsive on other platforms. In this case, you also have my contact data.

  3. My mental health has been harmed because the documentation is so bad, and I've spent too much time and energy spinning my wheels, that's both psychologically draining and harmful to my mental health. I fully understand the issues you're having and. I think it's possible that this code from the prior maintainer who was gaming coverage metrics but still writing bad code might have done it to you. I'm not blaming you.

  4. I personally consider needing to use that sync command to be a worst case scenario. Maybe it's a feature request, but I expected to be able to hit an API endpoint that would just do this for me so that once I created all the products and everything on the stripe side, I could just click a button in the admin or whatever and it would have my plan and subscription data available so that I could then figure out how I want to display it or format it on the "buy my stuff, here are your options" page.

Take https://dj-stripe.dev/usage/manually_syncing_with_stripe/ For example, it makes it look like it's going to explain something and then it's just blank, empty space. Yes I get that Customer.sync_from_stripe_data(customer) will sync that object (its badly explained, but I know what a classmethod is and figured it out), but Trying to create a new customer object that is fully synced and attached to a specific user that includes metadata or just simply has its fields populated well is a complete pain, and the djstripe code itself doesn't seem to support it, You literally have to create the initial object with the DJ stripe code oddly, and then use the customer ID of the generated object to send another non-dj-stripe stripe request to modify it, and then you still have to manually sync it. Thats not optimal.

jleclanche commented 4 weeks ago

Maybe it's a feature request, but I expected to be able to hit an API endpoint that would just do this for me so that once I created all the products and everything on the stripe side, I could just click a button in the admin or whatever and it would have my plan and subscription data available so that I could then figure out how I want to display it or format it on the "buy my stuff, here are your options" page.

The happy path is that you have webhooks already set up at this point, so this will happen naturally. The manage.py command is indeed only used in worst case scenarios as dj-stripe will proactively fetch missing objects if it finds them (eg. in linked objects). There is an admin command to sync/refresh stripe objects. The full sync that the manage.py command does is prohibitively expensive and should not be run unattended.

but Trying to create a new customer object that is fully synced and attached to a specific user that includes metadata or just simply has its fields populated well is a complete pain, and the djstripe code itself doesn't seem to support it, You literally have to create the initial object with the DJ stripe code oddly, and then use the customer ID of the generated object to send another non-dj-stripe stripe request to modify it, and then you still have to manually sync it.

The procedure here would be Customer.sync_from_stripe_data(stripe.Customer.retrieve("cus_XXXXXXXXX")). That is, retrieve the object from Stripe, then get the data from it into the db via sync_from_stripe_data. It would be nice to have a single method to do this for convenience though (PRs welcome on this).

You seemed to have misunderstood the intention of that email. You literally offered me money to help you, but you didn't explain how that was going to work as I have no company set up and actively avoid income from other nations due to the tax liability. I was simply doing due diligence and asking follow up questions.

I'm happy to resume that discussion any time; I responded to your email and was waiting for a follow-up from you. But it sounds to me like this is a no-go, and I'm not going to accept help if it means putting undue stress on you.

kavdev commented 3 weeks ago

I think it's possible that this code from the prior maintainer who was gaming coverage metrics but still writing bad code might have done it to you.

easy now, lol. If you want to call one of the maintainers out on something, come with receipts (and file an issue with them).

At that point in time, our docs were pretty robust, but only because the surface of stripe objects was low. Since then, we've focused our limited resources on keeping parity with stripe and implementing stability improvements.

This sounds like we need docs for "How to manually sync objects". That's on us (well, at least me) for assuming that everyone is using webhooks to keep their database in sync with stripe.

duaneking commented 2 weeks ago

This sounds like we need docs for "How to manually sync objects". That's on us (well, at least me) for assuming that everyone is using webhooks to keep their database in sync with stripe.

No, the error was in assuming stripe would never make a mistake or not have delays. Taking on the stripe API as a dependency just means we have to consider the stripe API to be something that can error as well. The webbooks are not always reliable, and so are not going to always fire immediately, thus they can not be fully trusted in that way to be perfect. Due to these gaps in the stripe infrastructure, its critical that we have the ability to force the sync of data. Having the ability to force a model sync also allows user scenarios and testing to be more flexible.

That code is very simple and it's just not documented well. The code is simple and just a real pain to get summed down as it requires research to ultimately get down to:

import stripe
import logging
from djstripe.models import Price

def sync_model(remote_cls, djstripe_model_cls):
    for stripe_obj in remote_cls.list():
        if stripe_obj:
            logger.debug("sync_model called: obj=%s", stripe_obj)
            djstripe_model_cls.sync_from_stripe_data(stripe_obj)

sync_model(stripe.Price, Price)

A proper solution would not just be to document the API call, it would be to provide some default utility API views:

  1. One for logged in admins/staff only that syncs Product, Plans, Prices when hit (I had to build this myself so I could work on the same codebase from multiple machines and initialize a dev environment faster)
  2. One that allows the current logged in user user to make an async request to have their stuff re-synced. (I have not built this)

The current docs also make it clear that the get_or_create stuff is unduely limited, slowing down attempts to stay in compliance by initially creating the customer with the correct information. As noted above, you have to do a long song and dance to get this updated, and its a real PITA.

        customer, created = Customer.get_or_create(subscriber=user)

        if customer:
            if created:
                # generate new metadata
                user_metadata = create_stripe_customer_metadata_for_user(user)

                # Update the customer with the new metadata
                old_customer = stripe.Customer.retrieve(customer.id)

                updated_customer = stripe.Customer.modify(
                    old_customer.id,
                    metadata={**old_customer.metadata, **user_metadata},
                    description=user.username,
                )

                if updated_customer:
                    Customer.sync_from_stripe_data(updated_customer)
onekiloparsec commented 2 weeks ago

I don't know how many objects you have to sync @duaneking, but after a few years of using Stripe's webhooks, I can confirm it is very reliable.

Moreover, you can easily customize which events are going to which route, you can replay them, see the details of all of them. My error rate is actually of 0%.

If dj-stripe had to provide one feature, that would be this.

But I think we are loosing a bit the sight of our common objective. Docs are not perfect, okay. Everybody has lost some amount of energy understanding how to manage the thing, okay. I am not trying to minimize nor ignore everyone's difficulties. I simply think, they are also balanced by the regular and patient work that has been done, until now.

That, being said, could we agree on putting efforts on fixing the docs? As far as I understand, the v3.0.0 is in preparation. Can you confirm @jleclanche? Even if this is in a remote future, that could be the right "canvas" to fix that docs one for all, isn't it?

jleclanche commented 1 week ago

@onekiloparsec Absolutely, I want docs to be fixed for 3.0. Nevertheless, I'll be working on an intermediate 2.9 release which will include some bridge changes as well.

abe-101 commented 1 week ago

Little update it seems that both dj-stripe.github.io/dj-stripe and dj-stripe.dev/dj-stripe now redirect to https://dj-stripe.dev/

This will help with google search related confusion - but does not yet address that fact that the build process for the docs section on https://dj-stripe.dev/ fails and is therefore outdated

abe-101 commented 1 week ago

docs.dj-stripe.dev Seems to be the last hanging fruit. it it stale at 2.5. My guess is the domain docs.dj-stripe.dev is set up to point to the readthedocs site dj-stripe.readthedocs.io

@jleclanche can you take a look at the dns record for the docs subdomain?

abe-101 commented 1 week ago

@czue

@jleclanche the text "Stripe Made Easy for Django Developers" does not appear anywhere outside of an html file on that branch or on the other branch. I cannot find any evidence of where it is coming from. Is it right that the current state is that nobody who is still working on the project knows how that site is built? Would @apedrob be able to weigh in?

for what its worth i think the text originates from the mkdocs.yml file located in the dj-stripe.github.io repo.

Here is a perm link to the original commit https://github.com/dj-stripe/dj-stripe.github.io/blob/b92a7538d16135ee06235003f560f33459ddc42b/mkdocs.yml#L3-L3

site_description: Stripe Made Easy for Django Developers

and it seems that the yml config file is then referenced in the home.html in the overides folder https://github.com/dj-stripe/dj-stripe.github.io/blob/a6c902a2304dd8e59ee9b9de490a4221cc5c816e/overrides/home.html#L13

      <h2>{{ config.site_description }}</h2>