Embrace Open Source culture: the 5 common transformations.

Article originally published at Linkedin on May 15th 2016.

This is a story of what I have lived or witnessed a few times so far. A story of an organization that used to consume, develop and ship proprietary software for many years. At some point in time, management took the decision of using Open Source. Like in most cases, the decision was forced by its customers, providers, competitors... and by numbers.

A painful but unavoidable transformation was required.

 

1.- Open Source consumer

Engineers had to learn a new system, adapt or re-write those features that used to made the organization unique, together with many other painful actions. It was expensive at the beginning but, due to the cost reduction in licenses and the change that Linux represented in the relation with providers, in a few years it was clearly worth it. And if it wasn't, it didn't matter since it was what the market demanded. There was no way back

Every software organization has gone through their unique journey, but the final sentence of the story has been the same for all of them: they became Open Source consumers.

 

2.- Open Source producer

This organization gained control over its production and, by consuming Open Source, it could focus many resources in differentiation, without changing the structure, development and delivery processes. At some point, it was shipping products that involved a significant percentage of generic software taken “from internet”.


It became an Open Source producer.


You can recognise such organizations because they frequently create a specific group, usually linked to R&D, in change of bringing all the innovation that is happening "in the Open Source community" into the organization.


Little by little this organisation realised that giving fast and satisfactory answers to their customer demands became more and more expensive. They got stuck in what rapidly became an old kernel or tool chain version.... Bringing innovation from “the community” required back-porting, solving complex integration issues, incompatibilities with what your provider brings, what your customer wants.


So they have to upgrade.


This organization will be able now to take advantage of all the common features and compatibility that the new kernel, the new tool chain... brings. But, guess what, forward porting all the differentiation features this organization has developed, all the bug fixes, is so much work and so complicated that the challenge put the organization at risk.

 

 

3.- Open Source contributors

The organization feels now the downsides of becoming a blind Open Source consumer and producer. Execs feels like when a bubble explodes and they are inside of it. They has less control than they thought, which turns out to be expensive, and what is worse, they lack the expertise within the organization to gain it....

But, after struggling for some time, this organization survived, which means that it has learnt some lessons:

• Upstream those features that are not differentiation factors any more.
• Increase the investment in that reduced groups of rock-stars that are up to date of what's going on "in the different communities".
• Invest in those Open Source projects that develop the key software you consume.
• Even better, start your own Open Source project to promote your technologies and be perceived as a leader...
• Reduce the upgrade cycle, so the "upgrade pain" is lower. As a side effect, the organization has the opportunity to increase the cash flow when doing two smaller upgrades instead of a big one. At the very end, the real profit comes with the first update, not with the new version, right?

This organization ended up upstreaming features when they could, normally very late, because “they do not have time”, frequently assigning that task to young inexperienced developers or, even "better", subcontracting it, which is "cheaper".

You can recognise that this organization has gone through the described process when attending to conferences given by any of its executives. They cannot stop talking about how much they contribute to this and that community, about how awesome the community is, how important it is to be open and share.... they are referring all the time to communities/upstream as "us” and “them". They think they got it, they really do.

Most of them believe they are in the crest of the wave after going through this third transformation process. They are innovative, they has been able to reduce their time to market, they are gaining reputation within a variety of communities… They are not just Open Source consumers and producers any more. They are also contributors. Some of them even heavy and "successful" contributors.

But if you look closer, they have not adopted "the Open Source way".

This organization keeps their traditional processes intact. It is managed in the same way. Decision processes are taken like when it was a proprietary company, it has not improved transparency significantly, it does not share code and practises among departments.... there is a totally different reality in front and behind its firewall, between production and R&D, between management, engineering, customer support, etc..

This organization face friction because of this reality. It still cannot move fast enough. Upgrading is still too expensive since now they have to do it more often than years ago, upstreaming goes so slow, when it happens. They cannot control the communities they are investing on...

So going through a forth transformation becomes unavoidable. Some refer to this transformation as "upstream first".

 

4.- Upstream first or becoming a good Open Source citizen

This fourth transformation basically means that upstreaming is part of your development process, not an aside task. It also means that communities are part of your delivery strategy, not an after market topic, that R&D is a two way road where you do not just consume innovation created by others but you share yours, not just "promote it". You really need to get involved.

This organization will learn that by becoming more open, their engineers learn more and faster, so the organization itself. It is at this stage where the organization really understand where the real value is in the software they produce compared to what is commodity...or that is what its executives and managers most likely believe, once again. :-)

But open source (no capital letters any more) is not about being open, but about being transparent, which means that is not just about seeing what is behind the glass, but also understand it.

I believe the fictional organization I am talking about will have to take one more step, the fifth one. It will be about "becoming upstream".

 

5.- Becoming upstream or being an open source organization

This is about understanding that, if you consume, produce and contribute Open Source, the smart thing to do is becoming an open source company. I think it is naive to pretend taking full advantage of  Open Source while keeping your traditional corporate culture, which collides with the one of those who produces most of the software you consume and ship, who are your “upstream”. You are building your business on top of them. Since you cannot control them, become "them".

The smart thing to do is to surf the wave, not fight against it, generating friction. Any manager knows that friction is expensive, reduces focus and drives away talent.  It is bad for the business.

The required culture change to succeed in this fifth transformation involves thinking less about us (company and customers) and them (community), and more about us (ecosystem). It can't be any more about upstream and downstream but about technology and service. It has to be less about upgrading and more about updating, less about "manage" and more about "lead" at every level of the organization, not just referred to execs and managers.

It is a transformation in which engineers are empowered, where management is more focused on collecting information for execs instead of producing it, and after decisions are taken, their key focus is alignment. A transformation in which execs get closer to where the real value is, to people, because they are the "masters". A transformation in which engineers not just follow, they get exposed, they take responsibility and assume the consequences... getting paid for it.

An environment in which accessing to key information does not depend on your position within the organization chart, which means that power does not depend so much  on what others ignore, but decisions are taken based on shared knowledge. A culture in which transparency is the norm not the exception.

In summary, a transformation that leads to a stage in which the organization steers its ecosystem instead of driving it. So it leads it in a sustainable way.

 

A quimera?

I understand it might sound like a quimera, but:

1. No more than it would have sounded 15 years ago any of the stories that so many CEOs or Open Source Program managers from leading corporations are telling nowadays in popular FOSS events about "their transformation".
2. I do not think the debate is if this fifth transformation will be needed, but about when and how to go through it.
3. My +10 years of experience in Open Source and +17 as manager tells me that, waiting to face any of the first four transformation processes until you have no choice is an unnecessary risk. I suspect the same will apply to the fifth one.

So my message is,

1. Consume, produce and contribute to open source being a good citizen.
2. Embrace Open Source culture... better sooner than later.

Pic link.

Testing => quality. Really?

Introduction

Nowadays the topic automated testing is becoming mainstream. Organizations and projects are investing significant effort in creating tests, using tools to automate them and plug them in their delivery chain. Combined with continuous integration tools, automate testing increases the usefulness significantly. I obviously find this trend unavoidable. Sooner or later every software organization will eventually go through it, if they have not already.

This movement is fairly new. Concepts like automate testing or continuous testing, in the context of continuous delivery, still do not have 10 years of history. We need to be careful with trends. The topic is so hot these days that the association between automated testing and quality is becoming the norm, also in Open Source.

Open Source became the winning "culture" in several industries more than five or ten years ago. Automated testing in the context of continuous delivery was not popular back then. Still, Open Source influence and adoption expanded also because of superior quality.

How come?

When I think about quality in Open Source, one key principle and three actions come to my mind.

 

Principle: transparency

Transparency is about seeing what others are doing, but also about understanding. This second part is too often forgotten.

Action 1: Code review

Transparent code review, (again, see & understand) is, in my opinion, the most powerful quality assurance measure a project or organization can apply. It is the fundamental action in what some call the FLOSS development model.

It has a side effect that I really like as manager: it improves younger developers skills. It also brings many other positive side effects.

 

Action 2: dogfooding

A few weeks ago in a workshop with a customer, Codethink CEO Paul Sherwood was explaining this point with an example that I stopped talking about several years ago. I found it so obvious that at some point I gave up fighting for it. After listening to him, not anymore. The example was.... your organization is developing Linux based products, use Linux, not Windows.

Simple, right?

Dogfooding is another of those actions that in long term Open Source projects is frequently taken for granted but that is not the norm in commercial environment. So many projects driven by newcomers to Open Source do not pay enough attention to it.

The impact over quality of dogfooding in the mid term is impossible to calculate. Still I believe is huge.

Action 3: delivery model that maximises the influence of early adopters

Who are early adopters? They are the developers or power users who like to consume experimental or pre-releases of your "product". The number of those willing to report bugs is significantly bigger in relative numbers than in consumers.

Increasing the number of early adopters, reducing the hurdles they face to use your software, analyse/debug problems and report should be a key activity among those projects worried about quality assurance. Adapting your delivery process to maximise their impact, not just have a positive effect in the use cases your software was designed for, but in others, expanding the knowledge about how your software will behave in the hands of users. Like it should happen between developers and delivery engineers, the feedback loop with early adopters should be very short, so you can provide them improved pre-releases in short cycles.

Open Source has reached the current point understanding how important the role that early adopters play is.

Personal note about this third topic

I want to make a point here before moving forward.

It seems to me that there is a new wave of Open Source projects, specially those driven by commercial organizations, that underestimate the mid term effect early adopters have on the quality of a project. I also see how the Continuous Delivery hupe, focused on the developers and delivery engineers, is leaving the early adopters behind in some cases. Specially in those Open Source projects in which the project is developed and delivered by full time dedicated engineers.

Many projects pay little attention to making their frequent releases truly installable, documented, simple to debug without complicated tools or even centralised infrastructure, bug trackers simple/fast to use, treat bug reports as a valuable asset . In summary, early adopters cannot follow the pace and, when they do, they need to spend a lot of energy to be valuable.

Let's go back to the main argument.

 

Conclusion

Code review, dogfooding and early adopters in transparent environments has been, I believe, the pillars that has made Open Source what it is today in terms of quality. And then, only then, automate testing, or continuous testing comes to place, in addition, not in substitution, not before, not in between.... in addition.

Are you doing Open Source? Don't take shortcuts. Surf the "trend wave" instead of embrace it blindly. Learn first, look carefully what sustainable projects are doing.

Quality is as much as culture as it is about having a nice dashboard full of green lights. Testing => Quality is, in general, a wrong association of ideas.

And yes, test frameworks, board farms executing thousands of tests, green lights in dashboards, etc. are awesome. Probably a forth pillar in the coming future.

Say Hi! to the new GENIVI Development Platform

On Wednesday February 17th, the GENIVI Alliance released a QEMU image of the GENIVI Demo Platform ivi9 Beta version, together with everything needed (instructions, source code, recepies, etc.) to build GDP-ivi9 with Yocto. A few weeks later, on March 8th, the first release candidate was published.

Finally, last April 19th GDP-ivi9 was published targeting QEMU, Renesas Porter and RPi2. Check the release announcement and download the different images and source code from the GDP download page.

I joined the GDP project in November 2015, leading a small team of developers from Codethink with the idea of moving GDP from a demo platform towards a collaboration platform. In summary, going from +r-- to +rwx. 

What was GDP?

GENIVI Demo Platform was the compilation of middleware components developed by GENIVI integrated with Yocto or Baserock, based on poky, designed to showcase and test the work done by GENIVI's Expert Groups.

What is GENIVI Development Platform?

At GENIVI's 14th All Members Meeting (AMM) is was announced that GDP would change his name, from Demo Platform to Development Platform, reflecting the new spirit that has arisen during the delivery of the  GDP-ivi9 version.

The general idea will be to mature those GENIVI's modules that were developed as proof of concepts (PoC) and provide up to date software together with a SDK, to attract developers to participate as contributors, having GDP as their number one Open Source platform for automotive.

Find further information about GENIVI Development Platform at GENIVI's public wiki, in the GDP project pages. The name change, recently announced will be reflected in the wiki in the coming weeks. 

Coming actions

During the coming weeks, the GDP delivery team will focus on the following topics:

• Migration from the current infrastructure to Github.
• Confluence will remain as the project wiki and JIRA as the ticketing system. The same applies for the rest of GENIVI.
• Add to our current targets another board: Intel Minnowboard
• Define together with the GDP community the roadmap for the next GDP version.
• Create a first alpha of the new version including the latest GENIVI software.

Feel free to propose enhancements or new features to GDP. The only thing you have to do is create a subtask under the ticket GDP-154, describe it and explain the benefits and potential risks/challenges. We will discuss them through the mailing list. I am looking forward of seeing Plasma 5 as part of GDP.

GENIVI 14th AMM and other events to promote GDP.

After te release of the new version, GDP maintainers and myself have been concentrated in making sure GDP was ready for  GENIVI's 14th All Members Meeting (AMM), that took place in Paris from April 25th to 29th.

I participated as speaker in 3 sessions and my colleagues at Codethink delivered a couple of Hands on Sessions about GDE-ivi9. It has been a lot of work but a good finish line for this release cycle. We will publish the slides the coming days.

A few weeks earlier I presented the GDP project at the Embedded Linux Conference (ELC), that took place in San Diego from April 4th to 6th. It was my first time at this conference and I enjoyed it. I also participated at the Collaboration Summit, invited by AGL and the Linux Foundation. I will provide some more details about these events in a later post.

I plan to attend to QtCon to promote GDP among Qt/KDE developers and to the Automotive Linux Summit, that will take place in Japan, to spread the word about this open project for automotive. I have also confirmed my presence in June 2nd at the OpenExpo, in Madrid. It will be my first event in Spain in quite some time.

Summary

It has been a very busy 6 months but very productive. Leading a small but promising Open Source project, that might have a big influence within automotive in the future, working together with my colleagues at Codethink and GDP community members, has been very interesting. I am learning a lot about this industry...by doing.