Modernizing Citrix Studio

Founded by former IBM engineer and principal OS/2 designer Ed Iacobucci, Citrix traces its roots from the early days of the computing industry. At the time, as companies transitioned to using microcomputers- there was a limitation to OS/2 that couldn't support multiple users on the same system.

Citrix's main product was a multi-user extension of OS/2, offering an additional go-to-market for Microsoft, opening a new customer base that helped them gather adoption from large corporates. With Citrix Multiuser, users could connect and simultaneously run character cell-based applications from remote terminals. Beyond the eventual IBM/Microsoft split, Citrix would extend MS-DOS and Windows NT-based systems. As enterprise computing moved beyond the mainframe era- advancements in hardware operating systems gave way to a technology that allowed companies to take better advantage of the hardware they owned: Virtualization.

In the computing space, Virtualization is a method of dividing the resources of computer hardware. Initially intended as a term dividing the resources that mainframe computers had, the word primarily references software-implemented computing environments. Platform virtualization is where administrators can deploy a virtual machine on a host computer with the full functionality of a physical machine. Admins adopt Virtualization to benefit from saving money on physical computers by having multiple VMs provisioned on a single server; it provides failover if those machines lose state and potential emulation support.

With a slew of acquisitions made in networking, business operations software, and software virtualization- Citrix's products evolved from mere extensions of Microsoft's Operating Systems to a suite of tools that IT admins use to provision and deploy virtual machines. In addition to its flagship virtualization product suite, Citrix also provides solutions for mobile device management, project management, and networking.

To better understand Citrix's virtualization portfolio, admins have three major concerns that need to be solved. Access and control, or rather, how employees need to authenticate and be authorized to use resources within a company. Provisioning and deployment, how resources are provisioned and managed. Monitoring and maintenance, how admins maintain their working environment for end-users, and ensuring uptime.

Citrix Virtual Apps and Desktops is the main product that handles the corporate admin's needs. It has several features and products within the suite. Monitoring and issues are handled in an application called Director, the configuration in Studio. Depending on the admin's end goal- they can configure several products in the suite, such as Machine Creation Services that provision machines on behalf of the user. Traditionally, these bits need to be deployed on an on-premises infrastructure.

30+ years from Citrix's founding, the enterprise market shifted from on-premises, self-hosted infrastructure- to a cloud environment pioneered by AWS's Andy Jassy, Azure's Satya Nadella, and Salesforce's Marc Benioff. After a successful activist stake made by Eliott Management's Jesse Cohn, the company sold off its GoToMeeting properties to refocus the business in line with its core competency, IT Ops.

In 2017, after a slew of re-organizations and cost-cutting efforts- the virtualization product found itself at a crossroads. At the time, IT organizations found themselves in an adoption trap. Cloud vendors increasingly prioritized their subscription-based tooling yet still had to maintain 100s of locally installed applications and infrastructure for their users. In addition, VMWare and new entrants in the virtualization space were also going after a lucrative market where customers traditionally have low churn. Companies that adopted virtualization technologies handled 7-year upgrade cycles, and as such, they are a source of stable revenue for companies looking to increase their ticket size increase. As a result, Citrix Virtual Apps and Desktops were on the verge of losing relevance. Perceived as a legacy offering in the face of Amazon Workspaces and VMWare Workspace One: inaction meant that you were on a declining install base as more companies considered alternatives to on-premises managed infrastructure.

In parallel, the FOSS and open-core business models transformed the developer's skillset from proprietary skills to transferrable skills. The same was about to happen with IT Admins. Traditionally- CVAD requires specialized training to use and deploy the software stack. There is significant pain to provision those pieces, and customers responded positively to the prospect of not having to manage those pieces.

After some evaluation, the product team decided to move with a cloud-hosted version of its software called Virtual Apps and Desktops Service that entered GA in 2017. At the time, the team would host an on-premises version of the 'control plane' on behalf of the customer, enabling service integrators and large customers to focus on onboarding their resources without worrying about upgrading and maintaining the product itself. This hybrid approach would, in theory, give customers the flexibility to migrate to the cloud on their terms.

In 2018 after office closures and layoffs, the Virtual Apps and Desktops team rebuilt its team in the face of its market challenges. I worked on similar cloud modernization projects as an intern at Adobe and Bridgewater; I joined Citrix to help them integrate my experience in building administrative interfaces into the next version of Virtual Apps and Desktops.

Citrix was intent to transition to subscription revenue business rather than one focused on selling perpetual bookings. It was vital that Virtual Apps and Desktops Service would be one of the primary drivers of that transition as one of the first products in their portfolio to make that transition.

Initial customers were warm to the product's positioning, the on-the-ground experience was different. The console used a proprietary version of the Remote Desktop Protocol instead of a native web experience. In addition, quality issues were abound leading to numerous outages. Customers who spent time deploying and setting up their infrastructure asked to move back to the on-premise version of Virtual Apps and Desktops or, worse, leave to a competitor.

At the time, the NPS of this product was -60 and was damaging the brand and blocking new deals.

Studio, the configuration console, is the front door to that experience. There were multiple issues:

My mandate on the team was to make sure the product was healthy enough to survive the transition to the newer version of the management console, all while maintaining the business and expanding in the face of new challenges.

I maintain that my philosophy around product management is to make the product successful at all costs. A great product manager can unblock what is critical, defer what isn't, and enable every organization to function effectively. Some people write about the function as a matter of influence; I don't necessarily think that's entirely the case- you want to make sure you have an excellent working relationship with all teams. Still, there will always be tension with different departments as everyone is being scored differently on their terms. The end goal is to ensure that those disparate pieces work together to ensure the customer has the best experience possible.

After dealing with an outage on the first day on the job, I made it a point to make sure that everything was in place to talk to the requisite stakeholders and get an understanding of the product's presence. I later learned this transition effort had moved around a few times where engineering morale was low, sales somewhat cynical, and marketing felt that the product team could not deliver on messaging and positioning.

Immediately, I spent time learning about the product, using it, and making a note of the primary defects of the experience. Simultaneously, I scheduled calls with customers and teammates to learn more about the market we serve.

When products first start in the enterprise space- the first few customers usually dictate the roadmap for the product. This is helpful as the product slowly but surely finds product/market fit. However, the customer relationship was a sales relationship rather than with the product org. As a result, the sales team would escalate any issues immediately proportional to their ticket size; what eventually happened is that the two offerings of products had a divided engineering effort for on-prem and cloud. The result was a disparate service that didn't build for the general case; instead, specific customer needs were battling over each other.

There were also issues with service delivery with quality matters affecting customers. Outages that occurred were detected by the support team, which only reported outages. Hence, there was a slew of angry customers who wouldn't open a ticket but instead vote with their feet and leave the platform entirely. Before any ticket became prioritized, getting the team to shore up its delivery methodology was a priority.

I worked with the cloud platform team to draft outage run-books and work with support to uniformly help them with messages for the outages. To increase awareness and share our customers' pain, I advocated for Product Managers to be included on the on-call list to help drive responses to outages and increase user empathy. Meanwhile, for the engineers on the team, I worked with my manager to review features or product offshoots that could be in line for deprecation to free up engineering resources.

The issues were affecting customer happiness and came in when customer demand for features was exceptionally high. I worked with engineering stakeholders to increase telemetry to help monitor failures and error rates while working with the field to let them know what customers were facing after the deal would close.

We were outsourcing the testing process. The testing environment didn't resemble the scale of what our customers were facing. We spent an additional effort to mirror the configurations of our largest customers who were pushing the limits of our software better than we were able to. All the while, I was jumping on calls presenting to our most irate customers explaining what we were doing to make sure that the quality issues weren't going to present themselves again.

With any team working on an increasingly complex product such as Virtual Apps and Desktops, it's paramount that team members understand its users. After some conversations, I realized many of the engineers and designers were working without context. I made an effort to bring those customer stories front and center to make sure that our customers weren't abstracted away from their needs, but rather that they had a proper understanding of how critical Virtual Apps and Desktops are to our customer's workflow. It became a mantra within the team; if our products don't work, our customers can't work. We were able to drive uptime over 180 days from an abysmal 98 percent uptime to confidently establishing and meeting an SLA of 99.5. It was table stakes for many teams but showed that the teams had much room to improve to follow our offerings.

At the very end of this hectic period, I made it a point to communicate with every team to give them the business context and set a mission on the board. This mission statement helped tie every product decision and focus our efforts by contextualizing the needs of our users.

"An admin experience that is welcoming to new users, unobtrusive to power users, and performant where there are no major delays to loading or where users are unaware of how long actions should take."

While we decided to pump the brakes on new features, Citrix needed to honor arrangements with Microsoft Azure and Google Cloud Platform. What ensued through the later sprints was a careful balancing act across different priorities of the business.

The team needed to clarify why the product team was entering a hiatus on responding to new RFEs and delaying features. As the product experience improved, additional pressure mounted on the product teams as not all customers were dealt with severe issues and were rightfully unaware of inevitable fires happening elsewhere.

I made it clear to the sales and support teams that I was (and still) am generous with my time speaking to customers and encouraged them to pull me to any meeting possible. I learned that if you are upfront with the reasons you aren't picking a request, they will understand just so long as you know the pain points they are facing. At this stage, we identified ways on how we can better communicate with our customers through in-product messaging and blogs. At the time, we would travel to user group meetups and calls to listen and incorporate that feedback back into the roadmap. Although there was a painful transition to the web console, the goal was to make sure that we could eventually address those pain points while offering short-term respite.

We decided to bundle and market the short-term collection of performance improvements to customers within our roadmap deck to signal to customers and prospects that we were fully aware of the issues faced in the past. Doing so may have seemed counter-intuitive but was essential to building trust with the field and our customers; it was around this time on release where we were able to unblock the sales pipeline once we reached the requisite levels of stability.

The engineering team working on migrating the console to the web version was able to work on a suite of RESTful APIs that also solved many issues related to automation and integration.

By productizing the feature- we were able to deliver value incrementally. We could package things as soon as they worked, clearly communicate the expectations, and let our adventurous admins try out newer features. To help with this mindset of incremental improvement, I tried to spend as much time being where our customers were as they discussed their preferences of the product on Facebook Groups, Forums, Public Slack Channels, and Twitter to manage expectations.

Implementing this leaner process required a lot of upfront work from a process and management perspective. I helped communicate the importance of building a better release pipeline to shorten feature lead time from 2 months to 2 weeks. By increasing the release cadence, we were able to reduce the number of bugs and defects drastically.

In addition, the oft benefit of having significant feature development blocked is that it also forces you to be creative about improvements and how you staff those initiatives. We realized massive gaps in documentation, scripting and SDK awareness, and potential improvements to the licensing experience outside of Virtual Apps and Desktops. While the team was making sure to deliver on this massive rewrite of the service, I spun up initiatives with other groups that resulted in a net positive for both teams committed to the work.

As a result, we released a feature with a separate engineering team called License Usage Insights, helping our customers get broader visibility into their investment with the service, allowing Citrix to get closer to its consumption-based pricing ambitions.

Much was also accomplished with refining the product <-> design <-> engineering relationship throughout this period. We realized that by having the issue tracking system be engineering-focused, we realized that we were spending too much time arguing requirements when roadmap requests were handed to IC engineers. We also used this time to refine the Epic template and help engineering teams push back when work was unfairly forced to the UI/API team's backlog.

After this period of demonstrating value to our existing customers, we could launch a preview of the new admin experience to end-users. In doing so, we were able to unblock the feature roadmap for this new console. Any new development or features would now focus on the new console.

We still were a long way off from fully completing the new version of the console and found ourselves at another junction with the product direction.

Before my tenure, to avoid a painful cosmetic redesign, it was decided that the web interface should look and behave the same way as the existing console. This was based on the assumption that our customer base would look and serve the same user personas as before: Citrix Admins. However, the industry was changing, now DevOps teams and infrastructure teams were adopting IT management tools and were looking at Citrix as a legacy technology. We needed to adapt to this shift quickly.

We had a few options on the table of varying pain

I initially set up two criteria that helped the team narrow down the decisions and choices to make.

It immediately disqualified the third option, and at the time, we weren't aware of Citrix's impending re-branding. The good news is from the new preview; we started to be able to act on some of the feedback that our admins were telling us; we figured that we would be able to demonstrate more value to our customers by acting on the feedback on the preview console considering we weren't willing to remove the legacy experience anytime soon.

In addition, due to the unblocked sales pipeline, we were offering the opportunity for our largest customers to adopt Virtual Apps and Desktops through a program called Hybrid Rights. We started seeing admins running into scale limits of the cloud-hosted version.

Long term, we wanted our admins to eventually use the web console in its entirety but already spent up the last remaining pieces of goodwill in delaying anything further to the older experience. To make sure the newer and largest customers were happy- we allocated some resources to the legacy experience one last time. Working with another PM on the team, we were able to work on setting up published scale limits to communicate the expectations we had with our customers to make sure they can plan and mitigate accordingly. (Much in the same respect on how Azure publishes Subscription limits)

As with good things that happen with a product, you eventually need more infrastructure to scale that growth as it matures. It became opportune to start working on a framework of helping other PMs rack their releases on the service to help measure adoption and gather usage insights of admin behavior. Due to the old, non-web nature of the console, we couldn't implement click maps or user behavior on the admin portal. Making that effort then helps us now be more in touch with our users.

As we accounted for the persona shift, we realized that we could build tooling with our APIs that helped automate the migration from on-prem to cloud. From my prior experience with Docker and Terraform, we also decided to help develop a tool that would eventually be integrated with the UI that allowed admins to manage their infrastructure in a more stateful fashion. They wished to follow many of the same paradigms they were using for their IaaS.

Then, the pandemic hit.

What was then hard to justify a series of delays and internal improvement decisions proved to be rather prudent as customers lost physical access to their offices and data centers. Much was reported on the unprecedented cloud usage growth, but not many realize that VDI workloads were primary drivers of corporate workloads.

The emphasis on mission criticality of the admin portal and hybrid messaging and positioning turned to be a wise decision for the product helping it gain relevance. At this time, surveys had us with an NPS of 0, a far cry from the -60 we had but only showed that we were being tolerated but not recommended.

From the months that came from the pandemic, we pushed on with our original plan knowing that it was the right decision in hindsight. Our goal was to ensure that we tactically supported each customer's emergency migration while building the new version of the console. With the feedback we gathered, we would update the preview showing to our admins that there was continued velocity and investment in the product aside from addressing the needs of our admins as they scaled.

Throughout the later stages of 2020, we were able to release new functionality over time with a transitory approach subsequently. As each node was released, we were able to track the usage of our admins as they initially adopted the new web admin portal in an extremely limited fashion, to one where now our admins use the web experience entirely.

The newer portal reduced average interaction from 15 seconds down to 1 second each. Admins spend on average less time on the admin portal than the old one, and they report an NPS score of 32 when now using Virtual Apps and Desktops Service.

After developing and releasing the CVAD APIs, numerous admins now integrate our cloud service directly into their infrastructure. Customers can delight that the flexibility that Citrix offers integrates into their setup.

Although I can not publicly share the current state of the roadmap: we have set up a solid foundation for additional value for our customers.

Most of all I am proud of fostering an internal product culture that suits the needs of our admins and our end-users; we were able to create a new chapter for a product not only we can be proud of but one that enables work for all.