search

AMarinic92 / 4560-IA-Accessibility-Checker

GNU Affero General Public License v3.0
0 stars 0 forks source link

Look into which ML library/framework to use #15

Closed RozenNoureev closed 4 months ago

RozenNoureev commented 5 months ago
TensorFlow PyTorch
Pros: Pros:
great for deployment great for research
Excellent support for distributed computing Dynamic graph
High-level APIs which simplify building and training models More intuitive, python like syntax
used extensively for Image recognition powerful tools for computer vision
powerful natural language processing tools (GPT-4 built in TensorFlow) Also has libraries for NLP (GPT-3 built with PyTorch)
Cons: Cons:
steeper learning curve focus on research, more difficult to deploy
Static graph may not be as optimized for large-scale distributed training
more challenging to debug While it is growing, it is a smaller ecosystem than TensorFlow

Notes

  1. [Stackify][Stackify] Says TensorFlow is better for distributed Training, while [Built in][Built-in] says distributed training is easier in PyTorch.

  2. [Stackify][Stackify] Says TensorFlow is worse for visualization because of static graphs, while [Built in][Built-in] says TensorFlow is better because of TensorBoard.

Rozen's Vote

I think that TensorFlow is the better choice for us, for the following reasons:

  1. The primary reason, is it is better for deployment which is better for our use case. (That being said, PyTorch is better for research, from my understanding, because it's good for faster prototyping and experimentation. So if we feel that experimentation is more important than deployment then PyTorch might be the better option)
  2. Although, as mentioned in my first note, [Built In][Built-in] Suggests that PyTorch makes it easier to implement distributed model building, [Stackify][Stackify] suggests that distributed computing is more optimized in TensorFlow, and it is the slightly newer article.
  3. Both are good for Computer vision, and NLP, which are both important for our use case, so it is hard to make this into a point for one or the other. But apparently, GPT-4 uses TensorFlow, and that's the newer version of GPT, so point to TensorFlow?

Sources

[Stackify]: https://stackify.com/tensorflow-vs-pytorch-which-deep-learning-framework-is-right-for-you/ "Stackify" -> Written July, 2023 [Built-in]: (https://builtin.com/data-science/pytorch-vs-tensorflow) -> Written March 2023 [Real Python]: (https://realpython.com/pytorch-vs-tensorflow/) -> doesn't say when it was written, but there's a comment on it from 3 years ago, so I believe this is older.