search

prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
16.06k stars 5.38k forks source link

[native] Documentation structure, pending PR discussion. #19084

Open Fiooodooor opened 1 year ago

Fiooodooor commented 1 year ago

After having the consensus on naming convention and seeing that efforts are being made to document the project the natural next step is the documentation structure discussion itself. This would include:

Bellow is the list of pull requests and issues connected to or extending this topic:

@rohanpednekar @majetideepak @mbasmanova

CC: @psbell-meta @aditi-pandit @amitkdutta @xiaoxmeng

majetideepak commented 1 year ago

@Fiooodooor I started a discussion here https://github.com/prestodb/presto/issues/19077 I will close that and paste my thoughts here

Prestissimo README can benefit from some restructuring. I propose the following changes

Add a Presto Wiki page for Prestissimo design, Velox integration, and limitations. The Github README needs a table of contents and must be limited to build and development-related information. I made a PR for the second improvement here https://github.com/prestodb/presto/pull/19075 View: https://github.com/prestodb/presto/blob/058a057500b692eb2ee6be6388654d78373c0ea7/presto-native-execution/README.md

Below are the sections I feel the README should address. Build from Source Build using Dockerfile Development Create Pull Request Advance Velox Version

@Fiooodooor You seem to have another handle @Mionsz. Can you consider using one to avoid losing updates?

majetideepak commented 1 year ago

We cannot practically maintain and support all the Linux distros. As new Linux versions emerge, and the Velox dependencies evolve, it is possible that the setup scripts can go out of date. We should officially support and maintain CentOS Stream 8 and MacOS since CI tests these distros. The remaining must be in issues or in a WIki. For example, I wrote an Issue here on building Prestissimo on AmazonLinux2: https://github.com/prestodb/presto/issues/18919

bianhq commented 1 year ago

We cannot practically maintain and support all the Linux distros. As new Linux versions emerge, and the Velox dependencies evolve, it is possible that the setup scripts can go out of date. We should officially support and maintain CentOS Stream 8 and MacOS since CI tests these distros. The remaining must be in issues or in a WIki. For example, I wrote an Issue here on building Prestissimo on AmazonLinux2: #18919

As Ubuntu is also very popular in both servers and development environments, and we already have ubuntu scripts in the repo, it would be good if we could maintain the docs for Ubuntu. There is a new LTS Ubuntu release every two years, it might not be a big problem. However, maintaining it in issues or wiki also sounds good.

majetideepak commented 1 year ago

Let's keep the Ubuntu setup script in the repo since it's popularly used. We can mention in the README which setup scripts are regularly tested and which are not.

majetideepak commented 1 year ago

@Fiooodooor I started a discussion here #19077 I will close that and paste my thoughts here

Prestissimo README can benefit from some restructuring. I propose the following changes

Add a Presto Wiki page for Prestissimo design, Velox integration, and limitations. The Github README needs a table of contents and must be limited to build and development-related information. I made a PR for the second improvement here #19075 View: https://github.com/prestodb/presto/blob/058a057500b692eb2ee6be6388654d78373c0ea7/presto-native-execution/README.md

Below are the sections I feel the README should address. Build from Source Build using Dockerfile Development Create Pull Request Advance Velox Version

@Fiooodooor You seem to have another handle @Mionsz. Can you consider using one to avoid losing updates?

@Fiooodooor, @bianhq do you have any concerns or thoughts about my proposal? I want to keep the structure simple since we don't have the resources to maintain and keep docs up to date. We can look into updating the Presto documentation with Prestissimo in the near future.

bianhq commented 1 year ago

@majetideepak I think it would be simple and clear to have separate md files for different OS, just like the setup scripts. Putting instructions for all the platforms on a single page could make the doc long and hard to follow.

On the main readme.md, we can put the table of contents, introduction, common rules, links to the other doc pages, and maybe a quick start (e.g., for docker or MacOS).

majetideepak commented 1 year ago

@bianhq what is the need for separate md files for each OS? Can you clarify the instructions that are unique for each OS?

bianhq commented 1 year ago

@bianhq what is the need for separate md files for each OS? Can you clarify the instructions that are unique for each OS?

I like the way that we have a separate page for each OS. People do different things on different OSs. For example, MacOS users are not likely to set up a testing cluster using the command line to run queries on large datasets. And, if I am a newcomer to Prestissimo and I am working on Ubuntu, I do not want to see 'MacOS' and 'CentOS' everywhere in the README (like what we do now).

If we do not add the notes or subsections for different OS, we must ensure the instructions work on each OS, this is also not an easy job.

So I think having separate files for different OS is more clear, an example is the vTune doc: https://www.intel.com/content/www/us/en/develop/documentation/vtune-install-guide/top.html.

Anyway, this is just my personal opinion. If we can make a single page very clear or if there are overheads to maintaining separate files, I support having a single page.

majetideepak commented 1 year ago

@bianhq thanks for this feedback. I am fine with creating a subsection for each OS as long as there is no duplication. For example, the CLion and Intellij setup is the same on both MacOS and Ubuntu, so we need to keep a single version to simplify the maintenance. The command line is the same for both MacOS and Linux as well since MacOS overlaps with Linux a lot. If Windows was in the picture, or if there was an installer, things would have looked very different, and the motivation for a separate page would be stronger. For now, my preference is to maintain a single page and see how much OS specific instructions creep in. We can always have OS-specific pages later.