Scaling GenAI in Software Development: Key Takeaways from Our May Webinar

The dust is settling on generative AI’s place in software development. GenAI has measurably increased developer productivity with features like augmented code completion and documentation search. Now the challenge is to prevent the costs of running GenAI from wiping out the gains from using it. This is a challenge that we’ve seen many large organizations run into.

Coder’s Rob Whiteley, CEO, and Tim Quinlan, Senior Technical Marketer, hosted a webinar to show how Coder manages GenAI for developers at scale.

What they covered:

• Overview: GenAI in the industry
• Coder’s approach: If you can deploy it, Coder can manage it
• Demo: Deploying popular GenAI tooling
• Architecture: autoscaling AI/ML for developers

Adopting GenAI for software development

Solutions like GitHub Copilot and Google Gemini are boosting developer productivity to the point of significantly reducing development costs. To get the most benefit, we suggest that organizations consider a few issues when adding GenAI to their developer environments.

An organization should evaluate each solution’s features and capabilities. Each has their own advantages and disadvantages, so committing to a single GenAI provider could limit potential benefits.

And while features like code completion are straightforward, the real challenge with GenAI is governance. The goal is to deploy GenAI effectively for your developers while keeping cloud spend under control. For example, not every developer needs the boost in performance from expensive GPUs.

Coder flexibility means better choices for GenAI

Coder is purposely agnostic about GenAI. Like every other developer tool that Coder can manage, we let you choose the technologies that best suit your developers’ needs.

You can use Coder to provision GenAI for each project and scale it efficiently. Coder lowers costs, speeds up onboarding, and simplifies the integration of GenAI with other parts of the development lifecycle.

Deploying GenAI with Coder

In the webinar, you’ll see Tim give a live demo of deploying GitHub Copilot and Google Gemini with Coder. That includes provisioning these solutions through Kubernetes then automatically installing the appropriate extensions in VS Code, both native and in the browser. All the developer has to do is start a workspace to get coding and interacting with the AI.

Coder handles autoscaling too

Coder also handles smart scaling of compute resources for your GenAI solution. Cloud GPU is expensive, so using it efficiently lets you keep the benefits of GenAI. Tim shows how to autoscale with Kubernetes node groups and taints/tolerations. For even more control, he shows how to combine that with Coder Enterprise’s RBAC features. This lets you allocate only the resources that a specific developer needs while still providing a responsive experience.

Learn more

We make Coder to give you control over your developers’ experience and the computing resources they need. And Coder is flexible enough to manage these things without forcing you to commit to specific platforms or technologies, including GenAI.

If you want to watch the full webinar, dive deeper by registering here. You can also check out our repo with example Coder templates for GenAI.

Join our upcoming webinar, "Deploy AI/ML Development Tooling at Scale with Coder," on June 25th, 2024, to explore how Coder can enhance your AI/ML development process.