RFC: Name Change

[EntilZha]

I have long thought that the name ScalaFunctional is not that great, but so far haven't done anything about it. I think that this might be a good time to come up with a better name that suits what it does and direction of the project better. To be clear, the name change is for the distribution name (repository, website, PyPI), the import name will not change, because too many things would break.

I would like to detail where the name came from, why its not that good, and what would be desired in a new name by explaining the overall goals for the project. I plan on posting some name ideas later this week after more thought, but would like to get ideas from others as well.

At the end of the issue, I will explain logistically what the plan is to TLDR not break anything.

ScalaFunctional Name

Origin

• API heavily inspired by/copies the Scala collections API
• Second major source of ideas is Apache Spark, which is written in Scala
• Library itself facilitates functional programming
• Didn't think too much about name when making library since I primarily wanted it on PyPI to use it easily at the company I worked for (ie, not setup private PyPI server and share code across projects)
• Matched the import name functional
• functional on PyPI had/has been dead for a long time, but cannot be reclaimed. Since it is dead, a name conflict with it from import functional is unlikely, but not possible to claim the dead project's distribution name

  Why its not good

• This is a Python package, not a Scala package
• Users may not care/know that it is Scala inspired so its confusing
• Package is focused on data pipelines. This is pretty clear from lots of work to support various input/output data forms (to python collections, files, sql, pandas, and probably more later). The name doesn't highlight that
• Hurts discoverability by LINQ users, which is a fairly large segment

  Overall Goals and Direction

  1. Support creating data pipelines using functional programming concepts
  2. Provide read/write interoperability with all common data sources in the domain of data pipelines
  3. Improve aspects of the functional programming experience in python that enhance the first goal
  4. Let LINQ users seamlessly use the same engine from a familiar API
  5. Provide the above with negligible impact on performance, and the possibility of a parallel execution engine

Currently, the project is doing very well in supporting the first two. The streams/actions API is very complete, and more or less all common data formats/sources are supported (file compression coming in next release). The next possible targets would be SQL DBs with SQLAlchemy or similar to to_pandas, provide a way to make an SKLearn node (auto generate class that satisfies the node API).

I am not quite happy with progress on the third goal, namely making lambda calls more succinct. This is my motivation to at some point natively support something like _ from fn.py. This is paired with the exploratory work I have been doing on a SQL Parser/Compiler. With the code/understanding I have right now, something like below is looking pretty easy:

User = namedtuple('User', 'id name city')
seq([User(1, 'pedro', 'boulder'), User(2, 'fritz', 'seattle')]).filter('id < 1').select('name,city')

I am fairly confidant that as time goes on, the fourth goal will be better and better met.

The last goal has a few things wrapped in:

• Currently performance is good (tested the other day) amortized over larger collections
• However, there is no parallel execution support. This hasn't been done because it requires quite a bit of work, and lots of testing.
• Currently, seq forces an expansion of its input, I would like to provide a family of seq operations that behave slightly differently for particular use cases. seq will stay default, sseq (stream sequence) will not force expand its input, pseq (parallel sequence) will provide a parallel execution engine. sseq is fairly low hanging fruit of these

  New Name Goals

• Describes better what the package does: data pipelines, functional programming, chain functional programming...
• Does not provide confusion with inspiration source, but makes sense given them

  Name Name Requirements

• Must be available on PyPI
• Name is not too similar to existing python package

  Names Taken on PyPi

• functional
• functionally
• chainz
• pipeline
• linqpy
• linqish
• py-linq
• asq
• PyLINQ
• chain
• fn
• pipe
• datapipeline

  Plan

  1. Reserve name on PyPI (don't release yet), rename repository to new name, change all references to new name, and place notices wherever needed
  2. Verify that old links redirect to new name (I plan on making a dummy repository to test this behavior)
  3. Make sure readthedocs, travis, and codecov work correctly with new name
  4. Dual release package under new name and ScalaFunctional as 0.6.0
  5. Current plan is to dual release under both names until 1.0, whenever that might be. The import name will not change, only the distribution/repository name. Open to comment on this or any part of the plan

Hopefully I didn't forget anything, open to comments on anything at all (including that name change is not a good idea)

[EntilZha]

Now that I had some time to reserve the name on PyPI today, would like to provide one possible rename option: PyFunctional

Pluses:

• Available, and reserved on PyPI
• Name is not too different from original name, matches with import name functional well
• Keeps link to Functional programming, keeps door open on this path

Cons

• Similarity: similar to functional, but thats already the case right now
• Doesn't refer to the pipeline/chain style. To have this, it would need to be in name

Other ideas:

• PyFun, PyFunc (taken)
• Pype (taken)
• PipeFun, PipeFunc
• FPipe

[ChuyuHsu]

PyFunctional sounds great, although the name ScalaFunctional we have now implies it's pipelining nature

[EntilZha]

Deciding on PyFunctional. Tomorrow I will be releasing 0.6.0 to put sqlite3 support on PyPI.

Summary of Actions

1. Everything in the repository will be renamed to PyFunctional with a note of old name
2. All supporting infrastructure will be tested, then the repository will be renamed
3. Release 0.6.0 on PyPI as PyFunctional and ScalaFunctional. This will be the last or second to last release under the name ScalaFunctional (unlike issue description in first comment, changed mind on this)
4. Separate from this, create a roadmap to 1.0. This will probably be in 2-3 releases from now. Planned inclusions are compressed file support, general sql db reading support, completed LINQ parsing for simple statements, a parallel execution engine, and possibly packaging an _ equivalent.

[EntilZha]

Name change is complete