Post

Open Source North 2024

Posted Updated
By Mark Norgren
10 min read
Open Source North 2024

Open Source North 2024

  • 1,299 registered and 1,212 attended

Session 1

Adapting your Developer Skillset today to an AI-driven engineering industry tomorrow

Ben Spain

  • 25 years as software engineer
  • Microsoft, TravelNet, Target, Thrivenet
  • Robotics Coach

Motivation

Skills across the Developer Journey

Skill #1 - Create Design Docs

  • Design document before started writing code
  • Clement Mihailescu - ‘To list potentially contentious aspects of the design and encourage debate’
    • Abstract
    • Changines to UX
    • …
  • Why Design Docs for AI?
    • Design Docs is the response to the Product
  • Requirements
    • Eventually the Design Docs will be the input to AI

Skill #2 - Use Diagrams for Presentation

  • Speaking with Diagrams
  • mermaid.js (Diagrams as Code)
    • state transition diagrams, sequence diagrams
  • Diagrams can improve the fidelity of AI generated solutions

Skill #3 - Frequent code reviews

  • eye for attention details, quality
  • need to be able to quickly scan
  • How to make your code reviewer fall in love with you
  • AI Training on Code Review Process Data
  • Eventually spot issues / patterns in AI generated code
  • You will be able to spot quality issues from AI submitted solutions

Skill #4 - Code in more languages

  • open to more languages –> recognize problems as patterns
  • Generalists are more productive than specialists
  • AI submitted changes should look like patterns
  • Human Skills for Programming: Pattern Recognition
  • You can review a larger variety of AI proposed solutions

Skill #5 - Adopt Test Driven Development (TDD)

Skill #6 - Quality gates for Shift-left testing

  • allow steady stream of AI submissions through initial checks
  • trunk based development - main is an automated conveyor belt
  • Inspection Rituals
    • full regression, manual clicking around, ‘warm fuzzy’
    • identify and get rid of this
  • Release
    • warehoused goods do not make money, they spend it
    • our job is to release a steady and consistent stream of value
  • AI submitted code proposals can be fully automated

Skill #7 - Ring based deployments

  • canaries, early adopters, all users
  • Codebases with AI submitted solutions can be trained on meeting criteria before rolling forward

Skill #8 - Feature Toggles

Skill #9 - Develop a dashboard with alerts

  • How do you validate that your code is working in Production?
  • Is your feature working in production currently?
  • instrumented code with metrics which will fire an alert….
    • sometimes more time spent on instrumenting and observing
  • Observability tools are making this easier
  • Observability is a team sport
  • Build a dashboard that the team is looking at
  • Alert the team
  • AI lives to monitor data
  • Continuous Delivery + Rapid Alerting = Less Manual Testing
  • Skill #9 - Develop a dashboard with alerts

Skill 10 - become part of your client analysis team

  • closing the feedback loop
  • get involved in customer insight
  • Understanding insights closes the virtuous cycle that enables for more AI generated solutions to be leveraged

Session 2

Empowering Engineering Teams with Large Language Models: From Ideation to Implementation

What is an LLM

  • Artificial neural network
  • Generates ‘human-like’ data
  • They are actually ‘creative’
  • Good at:
    • Content generation, language translations, learning and research, role-playing
  • Not good at:
    • Fixed knowledge base - static and limited
    • Understanding context depth
    • Conversational continuity
    • Dependency on user input

Story: A Product Journey Using Generative AI

Remote Patient Monitoring

Learnings & Takeaways

Luddite - Someone who is opposed to new technology or ways of working

  • Story about how in the 1700s they opposed new technology
  • Ned Lud - A mythical figure who was said to have destroyed weaving machinery in the 1700s
  • 1850 - Looms were destroyed, and the term Luddite was coined

  • 2024 - AI is the new loom

  • Companies that were once too small are now competitors
  • Right now, we are about 18 months since ChatGPT was released
  • Humans are still very good at seeing the big picture
    • Invest in being a generalist - solving valuable problems

URLs:

Session 3

Intro to Amazon Bedrock: Build a Data Foundation to Fuel Generative AI

Speakers

  • Chad Juettner
    • SDG Consultant
  • Paul DeLaria
    • LinkedIn
    • AWS Partner Solution Architect

Notes

  • AWS is customer-obsessed: what they do is driven by what customers are asking for.
  • What generative AI customers are asking for:
    • Which model should I use?
      • Models are constantly changing and evolving.
    • How can I move quickly?
      • Is my industry being disrupted? What are the trends?
      • Opportunity to be a disruptor.
      • Measure and understand risk.
    • How can I keep my data secure and private?
      • How do I protect my business?
  • Keeping up with the pace of change:
    • Uncomfortable state of understanding is probably where you should be.

Amazon Bedrock

  • SageMaker: More for data scientists building their own models.
    • Still there, but Bedrock is trying to democratize AI.
  • Easiest way to build and scale generative AI apps with foundation models (FMs).
  • Choice of leading FMs through a single AI.
  • Model customization.
  • Retrieval Augmented Generation (RAG): A way to augment the model with your own data.
  • Agents that execute multiple tasks.

  • Security, privacy, and safety.

  • Simplifies:
    • Choice, customization, integration, security, and governance.

Integration & the Folklore of the ‘API Mandate’

Meta

  • Meta Llama 3 (8B and 70B): Designed for devs and researchers.

Cohere

  • Enterprise generative AI and advanced multilingual apps with Cohere.
  • Command R: Generative language model optimized for long-context tasks such as RAG and tools, and large-scale production workloads.

Claude

  • Claude 3: Opus, Sonnet, Haiku.
  • Leading price-to-performance ratio.

Amazon Titan

  • Purpose-built models.
  • Indemnification for copyright claims.
  • Gives access to all the services a bit more.

Model Evaluation on Bedrock

  • Evaluate, compare, and select the best FM for your use case.

  • Common approaches for customizing FMs:

    • Prompt engineering.
    • Retrieval Augmented Generation (RAG).
    • Fine-tuning.
    • Continued pre-training.
  • Knowledge bases for Amazon Bedrock:
    • Knowledge bases now simplify asking questions on a single document.
    • RAG: A way to augment the model with your own data.
    • Agents that execute multiple tasks.

Enabling Semantic (Vector) Search Across Services

  • Semantic search across our services.

Agents for Amazon Bedrock

  • Execute multi-step tasks using company systems and data sources.

Guardrails for Amazon Bedrock

  • Custom safeguards.
  • Security, privacy, and safety.
  • Unbiased AI.
  • Unacceptable use policies.
    • Example: Sending customers to a competitor.

Demo

  • Chad Juettner did the demo.
  • topP: Same answer, different answer each time.

Questions

  • Amazon Q: Runs on Bedrock.

Session 4

Analyzing Basketball using Python, Elasticsearch, and Kibana

Jessica Garson

ELK Stack

  • Elasticsearch
  • Logstash
  • Kibana

Elasticsearch

  • What’s an index in Elasticsearch?
    • A collection of documents that have similar characteristics.
    • Each document consists of fields which are key-value pairs.

Packages Used

  • elasticsearch
  • pandas
  • matplotlib
  • nba_api
  • jupyter

Additional Information

  • getpass: Part of the Python standard library.

Session 5

Scalable and Resilient Orchestration: Process Automation as an Architectural Design Choice

Norbert Kuchenmeister

Scalable and Resilient Orchestration

  • Architectural Design Choices:

    • SOA (Service-Oriented Architecture)
    • Modular Monolith
    • Microservices
      • ‘Chaos’
      • ‘Complexity’
      • ‘One man’s microservice is another man’s monolith’
    • Serverless Functions

  • Modern architecture designs are good at optimizing resource usage.

    • Now diagram the business logic / data flow.

BPMN - Business Process Model and Notation

Event Queue Limitations

  • Loan Application Example:
    • Decision step relies on a credit check and a background check.
    • What if we don’t want a credit check if the background check fails?

Command vs. Event

  • Command: Do this.
  • Event: This happened (Fact).
  • Command-driven communication: Orchestration.
  • Event-driven communication: Choreography.

Considerations

  • Think about whether it’s a business process or a business rule.
  • Value chain, end-to-end process orchestration, existing systems, and new microservices.

Session 6

API Design First: Succeeding with API Governance

  • Travis Gosselin
  • Background
    • SPS Commerce
    • Distinguished Engineer working on APIs, developer experience, and API governance

Dev UX

  • What is Dev UX?

    • The activity of studying, improving, and optimizing how developers get work done.
    • Developers work in jungles, not planned gardens.

  • Dev UX == APIs

    • APIs are the universal language of development.
    • Over 90% of developers use APIs.
    • Almost everyone also builds APIs.

APIs Are Forever

Once an API is created, it should never be deleted, or changed. “Once you put an API out there, businesses will build on top of it,” Vogels said, adding that changing the API will basically break their businesses

https://thenewstack.io/werner-vogels-6-rules-for-good-api-design/

Approaches to API Design

  • Code First
    • More traditional.
    • Rapid prototyping, implementation-focused, team familiarity.
  • Design First
    • Spec first.
    • Generate code.
    • Consumer-focused.
    • Contract stability.
    • Collaboration.
  • API First
    • Write spec, then write/generate code.
    • Product requirements and capabilities.
    • APIs are treated as ‘first-class citizens.’
    • Everything about a project revolves around end product APIs.
    • API is the first thing you design.

Governance

  • How do we govern this?
    • ‘Modern roundabout’ designed to create a free-flowing, low-speed environment.
    • Safety is enhanced by X%.
  • API Design Meets Governance
  • API Standards
    • REST is an architecture, not a standard! Lots of ambiguity.
    • Just pick a way and do it.
    • SmartBear Report: Standardization is the top challenge.
    • RESTful is more what you’d call guidelines than rules.

API Style Guide

  • What is a style guide?

    • Informing our decisions so we don’t have to make them again and again.
    • Components:
      • URL Structure: Resource path, hierarchy, actions, query params, fragments.
      • Collections: Root elements, pagination, filtering, sorting, searching.
      • Serialization.
      • Request/Response.
      • Errors: Standard error response.
      • Naming.
    • Best Practices:
      • Less is more: Bullet points, clear statements, avoid suggesting implementation.
      • Living Document: Will mature over time, real-world examples.
      • Goldilocks Use Cases: Beneficial to not over-burden.
      • Align to Existing Standards: RFCs (e.g., RFC2119, RFC9457).
      • Examples!!
    • API Style Guide Examples:
    • Nobody really cares about your guide! Building it is not enough. You need to automate it!!

    • Every error response (4xx/5xx) SHOULD use the Problem Details RFC 7807.

      • Spectral

        spectral lint api.yml --ruleset .spectral.yml

Design

  • Effort required to design something is inversely proportional to the quality of the design.
  • Architecture Practice: API roadmap, API standards.
  • Product: API requirements.
  • Team Leads & DTLs: API design.
  • Teams:
    • API implementation.
    • API tweaks.
  • Review:
    • Consumers, stakeholders, producers.
    • Collaboration.
    • Not a gate, but a conversation.
    • Transparency.
    • Consistency.
    • Learning.

Tools

  • Accessibility to all stakeholders.
  • Integration with design tooling.
  • OpenAPI Domains: Models for 4xx responses.
  • OpenAPI Templates.
  • Spectral Linting: 42 Crunch, SwaggerHub.
  • Publish to SwaggerHub: Accessibility and cataloging.

Development

  • Contracts:
    • Enables mocking (Prism/Faker).
    • Enables stubs.
    • Enables SDKs.
    • Code samples.
  • Validation:
    • Build/Deploy.
    • REST API Fuzzer:
      • Microsoft Restler.
      • 42 Crunch.
      • Postman Contract Test Generator.
    • REST API Under Test.
    • Runtime:
      • REST API validation proxy.
      • Logging/report/enforcement.
      • Prism Proxy.
      • 42 Crunch.
      • OpenAPI-Cop.
  • Publishing:
    • Great API documentation: That’s a discipline that’s really lacking.
  • API Catalog:
    • Examples:
      • Stripe API.
    • Documentation Quadrants:
      • Tutorials, how-to guides, explanation, reference.
      • Examples are king!

API Governance

Resources

conferences
conferences opensourcenorth
This post is licensed under CC BY 4.0 by the author.