[DSIP-2][Doc] Refactor the document

[zhongjiajie]

Description

For now, we documentation is not easy enough to read and maintain. I think documentation is also an important for OSS, it would be

General

• [x] Sperate user_muanl/user_manal to multiple file with different level. https://github.com/apache/dolphinscheduler-website/pull/453
• [ ] Add directory for our image file, start this task until all we markdown document is done.
• [ ] Find a way to use codebase repo directory icon files instead of copy-paste

Content

DRY

• [ ] remove duplicate content in the document, for example in installation we have duplicate download mysql dependent
• [ ] Many parameters of task have same definition, find a way to handle it. maybe we could see what github/doc do, https://github.com/github/docs

Missing document

• [x] ~Chinese translation for ambari integration. @lukairui https://github.com/apache/dolphinscheduler-website/pull/494~ Our community going to drop support of Apache Ambari in the further, so we do not need this anymore.
• [ ] How to contribute to our document, including dev and older doc.
• [x] https://github.com/apache/dolphinscheduler/issues/6611
• [ ] 🔥 Missing Task: we missing some of our tasks in doc
  • [ ] SQOOP
  • [x] condition. https://github.com/apache/dolphinscheduler-website/pull/453
  • [x] switch. https://github.com/apache/dolphinscheduler/issues/6561
  • [ ] waterdrop
  • [x] pigeon. https://github.com/apache/dolphinscheduler-website/pull/453
  • [x] Task conditions screenshot https://github.com/apache/dolphinscheduler-website/pull/461
• [ ] 🔥 Missing alert plugin
  • [ ] Alert plugin dingtalk apache/dolphinscheduler-website#650
  • [ ] Alert plugin email
  • [ ] Alert plugin feishu
  • [ ] Alert plugin http
  • [ ] Alert plugin script
  • [ ] Alert plugin slack
  • [ ] Alert plugin wechat
• [ ] 🔥 Missing datasource plugin
  • [ ] Alert plugin clickhouse
  • [ ] Alert plugin db2
  • [ ] Alert plugin oracle
  • [ ] Alert plugin sqlserver
• [ ] Inside DolphinScheduler
  • [ ] How master node run
  • [ ] How worker node run
  • [ ] How master and worker community, and task lifecycle
• [ ] DolphinScheduler plugins development
• [ ] DolphinScheduler API
  • [ ] API docs and example, provider curl command line would be better.

To be classify

• [x] user_doc/globalParam https://github.com/apache/dolphinscheduler-website/pull/472
• [x] All the spi document for develpment. https://github.com/apache/dolphinscheduler/issues/6577
• [x] Unify dev-run and dev-quick-start https://github.com/apache/dolphinscheduler-website/pull/472

To be update

• [ ] For now, task name much unify in the project
• [ ] introduction.md
• [x] 🔥 Installation https://github.com/apache/dolphinscheduler-website/pull/465
• [ ] 🔥 Task Usage: we should add basic usage, example, notict about tasks
  • [x] #8016
  • [x] #8017
  • [x] #8018
  • [ ] #8019
  • [x] #8020
  • [x] #8021
  • [x] #8022
  • [x] #8023
  • [x] #8024
  • [x] #8025
  • [x] #8026
  • [x] #8027
  • [x] #8028
  • [x] #8029
  • [ ] #8030
  • [x] #8031
• [ ] Picture in English doc have Chinese content
  • [x] design，https://github.com/apache/dolphinscheduler/issues/7496
• [ ] architecture-design
  • [ ] Task queue. https://github.com/apache/dolphinscheduler/discussions/7835
  • [ ] Architecture diagram. https://github.com/apache/dolphinscheduler/discussions/7306
• [x] #7870

Syntax

• [ ] Use markdown table syntax instead of html table syntax <table>
• [ ] Find a better way to show our picture, for now it use html syntax in markdown file. I find our it can be show use markdown image syntax, such as this line

CI

• [ ] Add markdown link for our document
• [ ] Add chinese/english spelling ci for our doc
• [ ] Add document preview generation for each PR, example https://github.com/sqlfluff/sqlfluff/issues/808#issuecomment-787493894 . I think it really important to reduce doc build error, such as this PR https://github.com/apache/dolphinscheduler-website/pull/454, we get error dev leading page url make all our document error.
• [ ] Add doc style check for our documentation, inspired by https://github.com/apache/dolphinscheduler-website/pull/463#discussion_r734245581. maybe we could use https://github.com/tidb-incubator/zh.md

[github-actions[bot]]

Hi:

• Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.

[zhongjiajie]

Hi @lukairui, I assigned ambari integration to your now.

[lukairui]

Hi @lukairui, I assigned ambari integration to your now.

Done，please review my pr.

[davidzollo]

Do you have any suggestions about Docs? please leave message

[davidzollo]

I suggest to add doc about how to use open API it will be better if there are many parameters examples about each API interface

[zhongjiajie]

I suggest to add doc about how to use open API it will be better if there are many parameters examples about each API interface

Yeah, I think so. but I think we should find some way to auto-generate from our codebase instead of directly write doc by hand

[linjicong]

I noticed that some pictures are blurred

[zhongjiajie]

I noticed that some pictures are blurred

@linjicong Do you interested in this issue? I could create a new issue and assign to you if you like to do that.

[JinsYin]

We should also improve the document comments in the code, then use it in the website.

• https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#common-setup-options
• https://spark.apache.org/docs/latest/configuration.html#application-properties

[zhongjiajie]

We should also improve the document comments in the code, then use it in the website.

• https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#common-setup-options
• https://spark.apache.org/docs/latest/configuration.html#application-properties

You’re right do you have any idea to do it? Or you want to submit some PR to fix this?

[zhongjiajie]

The reason why we think the document is important, but why our community seems do not pay more attention on it

• https://github.com/apache/dolphinscheduler-website/pull/618#issuecomment-1006319922
• https://github.com/apache/dolphinscheduler-website/pull/618#issuecomment-1007123849

[Lucaszlei]

@zhongjiajie I am interested and able to help with the construction and improvement of the document. Next, I will gradually complete some of the problems mentioned above.

[zhongjiajie]

Hi, @SunnyZ-L Thanks for play attention to the DolphinScheduler documents, looking forward to your contributions and you could ping me if you have any problems.

[linjicong]

@zhongjiajie Is the file About_DolphinScheduler/About_DolphinScheduler.md standalone non-version module? This page now display 404 error when change language.

[zhongjiajie]

@zhongjiajie Is the file About_DolphinScheduler/About_DolphinScheduler.md standalone non-version module? This page now display 404 error when change language.

Yeah, I check and you're right, it seems we have error in our introduction section. Do you want to correct it @linjicong

[linjicong]

@zhongjiajie Is the file About_DolphinScheduler/About_DolphinScheduler.md standalone non-version module? This page now display 404 error when change language.

Yeah, I check and you're right, it seems we have error in our introduction section. Do you want to correct it @linjicong

I just submit a PR for it.

[zhongjiajie]

I add some independent new tasks for our tasks type usage. in https://github.com/apache/dolphinscheduler/issues/6535#issue-1026378395. You could search with the keyword "Task Usage" If anyone who interested in it, just feel free to take it and ping me in this issue.

[zhongjiajie]

I just submit a PR for it.

Thanks and I already review and left some commet in your PR @linjicong

[jj551]

@zhongjiajie architecture ! hi brother

[zhongjiajie]

architecture ! hi brother

@Kodalien I found out architecture has two children issue. which do you like to do?

1. Task queue. questions on architecture-design.md of 1.3.9 #7835
2. Architecture diagram. it seems that architecture diagram in the official website didn't update correspondingly #7306

[jj551]

Hi,brother@zhongjiajie have alreadly done,but I think new architecture needs to develop to demo in modification

[zhongjiajie]

Hi,brother@zhongjiajie have alreadly done,but I think new architecture needs to develop to demo in modification

Hi @Kodalien which part should add, could you tell more detail about it.

[jj551]

need 2.0 architecture diagram demo @zhongjiajie

[zhongjiajie]

need 2.0 architecture diagram demo @zhongjiajie

I got your point, @caishunfeng do you have time to update our architecture diagram?

[caishunfeng]

need 2.0 architecture diagram demo @zhongjiajie

I got your point, @caishunfeng do you have time to update our architecture diagram?

The architecture diagram in the document still applies to version 2.0.

[zhongjiajie]

need 2.0 architecture diagram demo @zhongjiajie

I got your point, @caishunfeng do you have time to update our architecture diagram?

The architecture diagram in the document still applies to version 2.0.

Thanks @caishunfeng for clarifications, cc @Kodalien it seem we could keep it in, WDYT

[QuakeWang]

Hi, I will do this task, please assign to me : )

[zhongjiajie]

Hi, I will do this task, please assign to me : )

Hi @QuakeWang, thanks for you attend to this issue. but it is a collection of some known document tasks, you should point out which sub issue in https://github.com/apache/dolphinscheduler/issues/6535#issue-1026378395 you are interesting in instead of whole issue

[QuakeWang]

Hi, I will do this task, please assign to me : )

Hi @QuakeWang, thanks for you attend to this issue. but it is a collection of some known document tasks, you should point out which sub issue in #6535 (comment) you are interesting in instead of whole issue

OK, I will leave a comment under the specific issue.

[EricGao888]

We need to consistently improve the quality of our docs, especially English docs, see: https://github.com/apache/dolphinscheduler/issues/11385