Sunday, May 22, 2016

Rx.NET community improvments

In recent months (early to mid 2016) there has been rising confusion and frustration with the lack of openness and direction of the Rx.NET code base. Code being open-sourced and hosted on GitHub in late 2012 was cause for celebration. The community could see the code without the need for a decompiler, they could raise issues and could they even submit code changes via a Pull Request.

Bright future

Over time, the community created issues and Pull Requests accumulated, but activity in the master branch seemed to come to a halt in Oct 2015. However anticipation was in the air as one of the members of the original team (and who I think is the lead of the Rx team inside Microsoft), Bart De Smet announced that Rx3.0 was underway at NDC in Mid 2015. Features seemed to include cross platform version of expression trees - Bonsai Trees. "We are going to ship this later this year" 19:33. The community was very excited.

Another key event was happening around the same time; Microsoft was making a large change to .NET. .NET Core was to be a massive undertaking that allowed .NET code to run cross platform, not just Windows, Windows RT and Windows Phone, but on Linux and OS X too.

On top of these key events, there was the growing organic momentum of the Reactive movement. A movement that in my opinion the Rx.NET team had a large hand in popularizing and setting the standard for. Rx had matured in the eyes of the development community. It had been ported from the original implementation in .NET to JVM, JavaScript, C++, Ruby and others. This widespread adoption across the industry gave developers and managers alike confidence in the tool.

Bright future?

However, as time passed new cautious developers looking to adopt Rx wanted to know that this technology wasn't at a dead end. With recent effective end-of-life for technologies such as browser plugins (Silverlight, Flash etc) and Parse, it is understandable why people with long term plans are looking for assurances. Even intrepid developers were looking for some assurances that things were still moving. After the announcement of Rx3.0 and it being inferred that we would see a change in late 2015, in conjunction with the .NET Core activity questions started coming.

In the gitter chat room for Rx.NET, questions about the next release started as far back as Sept 2015. Then in Jan 2016 more questions about the next release and if Rx is still a supported library. And again in February, March, April and May. The tone of the chat room seemed glummer and glummer each time the question was asked, and we realized that no-one from the Rx Team was answering. Even Tamir Dresher who was writing the next book on Rx couldn't get any response from Microsoft. To add to the feeling of rejection the community was feeling, we became aware of a private chat room that members of the Rx Team were engaging with a select few people from outside of Microsoft. While I was aware of one person in this chat who I would consider a key member of the community, most of the other people I would consider key members with a wealth of experience in Rx and relevant streaming technologies were left in the dark, excluded.

Just because we can't get a transparent and comforting answer about the future of Rx from someone at Microsoft doesn't mean the project is dead. However the 18 Pull Requests that have had no interaction from the Rx Team, and the lack of any commits to master in 7 months did leave a lot of people pretty uneasy about what was happening. Not only were we getting radio silence, but this was being contrasted with what appeared to be massive amounts of activity happening in the RxJs space.

Bright Future!

And then this tweet happened
The future of Rx .NET is still bright, and yes we are transitioning it to the .NET Foundation
To me it seemed to be a self congratulatory tweet. In my opinion, there wasn't much worth celebrating. Some people did some good work porting some Rx code to CoreCLR and did so on a private fork/repo. Not on a feature branch on the main repo where others could watch, but somewhere else. A heated exchange followed on twitter, for which I apologize. It is not the forum for an exchange like that. But it did prompt me to write what I hope is constructive feedback below.

A more collaborative and transparent future?

The Rx code base is open source, which I understand doesn't mean a free-for-all. However, I think it is reasonable to expect a little more from a project than what we are getting. Here are some things I think are reasonable expectations:

Be clear about the status of the Repo/Project

In the front page (Readme.md) have the status of the repo. This should probably include which version is the current latest, which is the current pre-release. Link to them in nuget and their tags/branches (like RxJs). Highlight what the roadmap is (like ASP.NET).

Instead of people having to ask in chat rooms, twitter and forums about the status of the project it should be right their front and center. Something like
We're hoping to release a new batch of functionality later this year, based on internal developments that took place within the Bing organization. As part of this, we're looking at doing future developments in the area of our reactive programming cloud platform in the open here on GitHub. Currently, we're working out the details to transition parts of the technology to the .NET Foundation.
Followed up with something like "Transition to the .NET foundation can be slow, so this may take til Q3 2016. After that we can start porting our internal work in Q4 2016. In the meantime we would love to get your help on the issues that are up for grabs." (quote my own and made up). Now we would all know what was going on. It would take 10min to update that readme.md file.

Be clear about how to contribute

Contribution is not just about signing a CLA (Contributor License Agreement). It should include some guidance on how to raise an issue, link it to a PR and create some dialogue to validate that the change is inline with the direction of the project. This should include which branch to target 'master', 'develop' or perhaps a feature branch? It should also set the expectation of what the project custodians will do for the contributor with regards to labeling, setting milestones or closing the issue.

Documentation

The readme, the wiki and the reactivex.io documentation is sub-par. Allow the community to add more links to other helpful resources. There are 2 PRs to update documentation, but they appear to have been ignored.

If this is Microsoft's documentation for Rx, then maybe the link should be front and center. However, it suggests that you "download" the Rx SDK from the Microsoft download center instead of using Nuget, and the samples kick you straight off in the wrong direction by sticking subjects in your face as the first example. I would imagine that there would be links to external resources like the various books and websites that are out there. A good starting point for the loads of options could be this compilation -http://stackoverflow.com/questions/1596158/good-introduction-to-the-net-reactive-framework/. A link to ReactiveX.io would make sense. On that note, the ReactiveX.io samples for .NET are incomplete. Even the basic `subscribe` and `create` methods are missing. 

Clean up the branches

It appears that master is the only real branch (and gh-pages). So can 'develop' and 'BetterErrors' be merged, deleted or documented?

Creating a tag that relates to the last commit that was used to create a package is a helpful thing to do. This allows people to identify bugs for specific versions and see if a fix is already in, or if they should consider raising an issue or a PR.

Engage with issues

In the last few days there appears to be a flurry of activity, however I think we can improve in this area. Issues seem to be one of three broad categories: "Acceptable", "Maybe Later" and "Not acceptable". For things that won't get accepted into the repository, let's be honest and transparent. Say this doesn't follow the direction of the project, but thanks for your contribution. Then close the issue, or give the author a few days for rebuttal or to close it themselves. For issues that look like they match to the current planned release or would be suitable for a future release then label it as such. The CoreFx team do a great job of this in their issues

Labels are not the only tool the team could use to communicate with. Milestones also are a great way to give visibility to what is current and what is for later. They also allow you to see how close to complete a milestone is. It seems that this was a prime opportunity to use them. Milestone 2.2.6 could have had just two issues. CI and CoreCLR build. These issues could have been labeled as Up for grabs. The community would have jumped at the task. However with 18 of the current 23 PRs having had no official interaction, you can see why the community appears aloof when the task is significant, but may not even get noticed.

As a concrete challenge to the Rx.NET team: Aim to close or label every issue in the Repo. 

Maybe create some new issues, and mark them as up for grabs. Watch the community jump to action.

Which brings me on to my last ask, which is basically the same as for issues, but for PRs. Engage. People that have raised a PR, probably have used the library heavily, found an opportunity for change, then forked and cloned the repo. From here figured out how to get it build (because it doesn't out of the box in Debug or Release targets). Figured out the code base, made a change and ideally some tests to support it. Finally they have submitted the PR. Best case scenario, I think would be 4hrs of their life they could have been doing something else. I imagine it would be quite a disappointing feeling to have 0 comments on your PR.

Symbiotic Relationship

I am not hating on Rx, nor am I hating on @ReactiveX quite the opposite. I think what the Rx (Volta) team have done is brilliant. I enjoy it so much I poured 6 months into writing documentation for it and a Christmas holiday writing a PR for it. There are others too out there that are equally as passionate.

But like in a loving family, sometimes you need to be the unpopular one and point out that hey, this is not okay.
All I am asking from the Rx.NET team is a little back-and-forth, a little transparency. I would think they have far more to gain from doing this than anyone else. I have worked with hundreds of people that have used Rx. A handful have contributed to RxJava and RxJs but none to Rx.NET, because it is too hard. I think there is a great opportunity for positive change here.

RxJS is almost 100% community run at this point with Microsoft helping steer the project

Can Rx.NET take a step in this direction? With brains like Bart's at the helm and the army of the willing already out there, then yes, we would have a bright future indeed. Just tell us what you want us to do.

2 comments:

scottt732 said...

I hope Rx has a bright future with a vibrant open source community in .NET. It is one of the most powerful pieces of the .NET ecosystem. I couldn't have learned it without introtorx.com and (the Kindle edition). I look forward to a response/good news from Microsoft. I'd love to see proper backpressure support, CoreCLR support, and even IQbservable+OData (see PushQA) in MVC 6.

Jim Wooley said...

Well stated. I have heard the same concerns multiple times when I've talked about Rx at conferences as well. Hopefully we can continue to grow support for the platform with some assistance from those who created it originally.