Tuesday, November 17, 2015

DockerCon EU Day 2 Keynote "Live Blog"

This was a "Live Blog" from the keynote this morning. Took me a bit to get it somewhat cleaned up and get access out to post this.

Ben on stage:

Recap of Day 1:
- #dockercon - #2 worldwide trending item on Twitter yesterday
- Keynote (using the Power of AND as a theme)

- Lessons learned on the path to production: custom scripts rarely scale, developers do not adopt locked down platforms, end to end matters for both dev and ops, build management & orchestration enables portability
- Ben talking about “Containers as a Service” - Build (Docker Toolbox) -> Ship (Registry Service) -> Run (Control Plane)
- Call back to yesterday and four layers of solutions - talking about creating a solution as an end to end flow
- Interesting that Run is called out as a Control Plane (and references Tutum on the next slide)
- 20% of all content pulled from Docker Hub is “official images”, but what about all the others? You know you can trust an official image. Project Nautilus was brought out to address this other 80%.
- Showing output of a Project Nautilus scan on the screen. It breaks down line by line each library used in a container

Docker Automated Builds:
- Talking about Automated Builds - 60k automated builds per week, 300% growth since January 2015. Automated Builds 2.0 is a rearchitecture of the system to address time and quality issues.
- New Build System uses per-Repo Dedicated Builders (you don’t share a build queue with anybody else anymore), starting a fresh build environment every time. This increasing time of parallel builds as well as guaranteed quality of a clean environment.
- Dynamic Matching is the other feature. Static mapping used before (you had to manually tag your builds), dynamic matching allows for variable based builds and more flexibility in the system over time

Docker Tutum:
- Now talking about Run phase (using Tutum) - Tutum guys on stage
- What is Tutum? - a cloud that allows code to production rapidly
Demo Time:
Talking about code from laptop into production - SaaS demo from yesterday (voting app)
- What will happen? Modify a feature, image created via Docker Hub autobuild, Image deployed in Tutum
- Showing Tutum visualizer - shows a visual representation of the app (both dev version and production version)
- The production version is deployed across regions in AWS as well in a private datacenter (balanced across both)
- Before they make a change to the app, showing the automated build in Docker Hub connected to GitHub
- Now modifying the application, commit to git repo, push to remote repo
- Showing Docker Hub changes and dynamic changes reflected from git
- Docker Hub builds the image and redeploys the image to production in automated fashion
- Take Away: Push to git and the automated workflow takes care of everything else in the build and push
- Now - to push to production from staging, Tutum shows a visual representation of the containers being upgraded. Production is upgrading in a rolling fashion automatically. “One click upgrade to production"
- What about resiliency in production? What is we take down a datacenter in production?
- Using the Tutum interface, wipe out a datacenter, Tutm redeployed the containers in a different datacenter and scaled back up to support the load (was actually a really cool demo)

- 3DExperience Company customer story slide on the screen now
- Customer on stage - Talking about consistency between development and production, simplification of tools for dev and ops, ability to deploy on their cloud, and the scalability and increased high availability provided by moving to Docker containers. This is a sneak preview of the results they have achieved.
- Showing a video of their product called HomeByMe (online 3D modeling of home improvements and planning) fully running on the new system
- The system has gone from concept to production in less than a year

Docker Universal Control Plane:
Scott Johnston (SVP, Product) on stage now
- Asked for raised hands on DockerCon - the vast majority (probably 80-90%) are first time attendees
- Asked for show of hands of who can’t put data in the clouds or can’t put control planes in the cloudss
- Production in the Cloud? Not for everyone due to compliance and security
- quoted Adrian Cockcroft “speed is the market share"

- Developers will always find a way to go fast, it’s their job
- We want Agility and Portability WITH Control
- This starts at the app level - How doe we know which images to trust, who signed an image and when, how to automate, etc.
- To support this, Docker Content Trust ad Docker Trusted Registry are now in sync with each other
- What about the Run aspect of all of this? What about the control plane?
- ANNOUNCMENT: Docker Universal Control Plane
- This was Project Orca - Integrated Stack for application deployment
- Self-Service App Deploys & Updates, Provisioning & Config of Heterogeneous Clusters, LDAP/AD Integration with Docker Trusted Registry, Native Docker API’s and CLI, Monitoring, Logging.
- Completes the end to end aspect of Containers as a Service

DEMO of Docker Universal Control Plane
- login to Docker Trusted Registry
- sign the app with Docker Content Trust
- push the app to the registry - show the app has been signed
- Now how to push it out and deploy it
- Flip over to Docker Universal Control Plane and login
- control plane sits on top of Swarm and integrated with Native API (to use Compose, etc.)
- Use Docker Compose to run the app - The control plane gives access based on LDAP credentials
- Control Plane auto detects the new build and adds it into the control plane dashboard
- Shows how many resources are being consumed per account, ops dashboard basically
- Now scale up the app by adding more containers to the voting app (from command line)
- Now talking about secret management to control variables and info
- Showing that secrets are based on the access control groups in LDAP (production is locked down vs. dev which is wide open)
- Now redeploy of the app using the secret to use that vs. the environment variables
- Control Plane allows you to roll credentials incase they are compromised, now do a docker compose restart
- Restrart and they showed the password has been changed and rotated

Docker Trusted Registry 1.4 is GA and Docker Universal Control Plane is 1.0 Beta as of today

Monday, November 16, 2015

DockerCon EU Day 1 Keynote "Live Blog"

Going to try something new here on The Cloudcast. It's been a long time since I did a blog, I'm at Dockercon EU this week and there was some interest on Twitter to get more info out about the keynote. Wireless was down during the show so this is a "semi-Live Blog". Might be some typos in here and this is a brain dump as things happened during the keynote.

- About 1500 attendees at the event
- Ben (CEO) on stage:
- Ben talking about Docker public image and that it is perceived as “just a developer tool”, they are much more
- Docker is about building tools of mass innovation - quote by Solomon

Stats Time:
- Docker has nearly 2000 contributors to the Docker project, over 10,000 pull requests
- global metope communities highlighted - 215 groups, 63 countries
- Over 60,000 project on GitHub have Docker in the title
- State of the Project:
- 240k dockerized applications, 1.3billion Docker Hub pulls, 5.6M Docker Hub pulls per day
- Docker has evolved from a container technology into an entire ecosystem of tools
- Open Container Initiative - 35+ members, 253 github forks, 130 contributors
- Docker used for stateful as well as stateless apps - really started as stateless and is growing into the other
- Docker in production - (see the DataDog study, a lot of stats used from that) - 8 surprising facts about Docker Adoption (google it)
- Docker in Production means making Docker much better and more robust. Must be portable and good for dev as well as ops, Secure and Extensible

Docker Stack:
- Solomon (Founder/CTO) up on stage now:
- Solomon talking about the Internet (lots of upgrades, doesn’t go down, ultimate at scale system)
- The biggest obstacle right now is software walled gardens, it stands between an eager developer and the Internet
- Docker is building an open software layer to make the Internet programmable
- Solomon talking about the Docker Stack - 4 layers in a building is the example
- Layer 1 = Standards. Let’s get everyone to agree on a way to interoperate
- Layer 2 = Infrastructure. The “plumbing” that enables everything to happen
- Layer 3 = Dev Tools. A collection of tools to help developer experience the best it can be
- Layer 4 = Solutions. How do you solve real word problems? What is the final answer? This is solutions

Docker Quality:
What is left after you ship a feature, Quality is making a feature work every time, for every user - Quality is security, reliability, handling failures gracefully
- What has Docker been up to? Quality tools for developers...
- first up, usability of tools, Solomon admits they have been working on usability of tools. Talking about docker compose right now, it is the “developer entry point” into the ecosystem. It is the must use tool for developers. As of the last release, can now do “magical” service discovery, can now use a micro-service architecture without rewriting code, and can now build persistent services with volume management
- Working on making the “little things” better for developers (virtual box integration issues, UI glitches, low priority bugs, better error messages) - lots of unglamorous work
- Working up to a story and a demo. Story of a developer on the first day of work. How soon could be developing an application? - Simple as download the Docker Toolbox and run one command.

Docker Security:
- Solomon talking about “usable security” - developers care about usability, not security. They care about security, as long as it doesn’t affect usability, otherwise they will just find a way around it
- How to give developers usable security? How do we move beyond Docker Content Trust and Notary?
- Docker Content Trust + hardare crypto = the ability to survive almost any key compromise (double layer of protection provided so you can rotate keys and replace as needed as long as the root key is kept safe)
- Announcement: Docker and yubico - hardware crypto key for Docker Content Trust
(Demo of the product) - plug the hardware key into the laptop, enable Docker Content Trust, docker push to Docker Hub, touch the key (physically) to prove you are a human and this isn’t a “bot” or something malicious, enter a password, done.
- LOL - made a backup copy of his keys and then published to github public - not a good thing
- Security team rotated the private key to prevent a compromise, tried the demo again and of course it failed because of key rotation. Was actually a very entertaining demo
- Take Away: With the right tools, any developer can become a secure software publisher
- Isolation of a container in Linux was difficult because so many things “make” a container. Over time this has improved. The last two left are really seccomp and user namespace
- The last two have been tackled in the Swarm/Engine experimental builds
- Huge question with a lot of different answers - “Am I running vulnerable containers?"
- Announcement: Introducing Project Nautilus - Built-in container security analysis in Docker Hub - trigger an automated scan anytime a container is pushed to Docker Hub
- soft launch 2 months ago, over 74 millions pulls to date already scanned, self service coming soon
- Benefits of this approach - Detect vulnerabilities regardless of the Linux Distribution, discovery of new vulnerabilities in Linux distributions and collaborate with communities to fix them, developers can use their favorite package manger (probably not the one that shipped with the distro)
- Take away: You can be secure without lock in to a specific distro

Docker at Scale:
- Next topic and Demo - Swam at scale
- Took the demo (Day 1 app and scaled this up to 1000 nodes in Swarm) - Now using swarm bench to scale this up to 50k containers across 1000 nodes. Once they are up and running, Swarm scheduler balances them across the cluster - real time this was done in less than an hour.
Note: Swarm tested to 50k containers but that was a limitation of EC2 right now. They expect to have better numbers in the future. Docker is dedicated to making Swarm the most scalable and usable system in the industry

Disclaimer: The Cloudcast was a media sponsor of Dockercon EU

Wednesday, July 22, 2015

The Periodic Table of DevOps Tools (by Xebia Labs)

Some things are so good that they just need to be shared. This interactive table (original) was created by Cloudcast alum XebiaLabs (Eps#182). It's not only a great way to represent all the tools that are being created, but it's clickable to give you more in-depth information on each item.

Nicely done, XebiaLabs team!

Tuesday, July 14, 2015

The Cloudcast #206 - Experience Building Large-Scale Clouds

Description: Aaron and Brian talk to Jeff Dickey (@jeffdickey; Chief Innovation Officer @Redapt) and John Griffith (@jdg_8, Software Engineer @SolidFire) about the evolution of Redapt, best practices for building large-scale clouds, comparing OpenStack to Docker communities and how the ecosystem is changing from Vendor to SP to VAR.

Links from the show:

Topic 1 - Tell us about yourself and some background on Redapt.

Topic 1a - You both have OpenStack background. Why are you here at DockerCon?

Topic 2 - Aaron knows Redapt from his day job, but you really got on our radar the past few weeks with a bunch of announcements recently (eg CoreOS Fest + Tectonic). How did Redapt get involved with delivering solutions around these new Cloud Native frameworks?

Topic 3 - What you do is really a next-step in how companies are able to build or consume these new Cloud Native frameworks. How does Redapt go about pulling these systems together?

Topic 4 - We talked yesterday about your team. How do you keep the talent levels up to date on your team?

Topic 5 - Redapt is well-known in the cloud circles. What best practices can you take from your learnings and apply them to all these Enterprise and Mid-Market companies that want to do all the cool stuff we hear about here at DockerCon?

Sunday, July 5, 2015

Free Ticket to OSCON - Tell us Your Open Source Journey

One of the coolest things about partnering with O'Reilly Media is that we get to give things to The Cloudcast community - including FREE passes to their excellent events.

For Velocity, we were able to give a free pass to Jordan Stone (@Cheddz) from Notion. He won our contest to tell us about his coolest project, and his Wireless Home Monitoring solution looks pretty cool to us.

For OSCON, we're giving away another free pass. This time, all you need to do is tell us about your journey to using open source software. Here's an example. Either send us a link via Twitter (to your blog or GitHub account), or drop us an email to show@thecloudcast.net. We'll pick the winner in the next week.

Interested in more great stuff from O'Reilly OSCON? 

Monday, April 27, 2015

Win a Free Pass to O'Reilly Velocity Conference

The Cloudcast is excited to announce a new partnership of O'Reilly Media! To kick things off The Cloudcast and O'Reilly have one free pass to O'Reilly Velocity to give away! We're also allowing our listeners access to free O'Reilly eBooks. Other great offers coming soon.

Velocity Contest details:
NOTE: Contest only includes the pass to Velocity Conference. It does not provide any coverage for Travel or Expenses - you're on your own for that.

We look forward to hearing about what you have going on!

-Aaron & Brian

Friday, April 17, 2015

The Cloudcast #187 - API Performance Monitoring

Topic 1 - Briefly about your background on the company and team (John was at Twillio and IFTTT).

Topic 2 - How is API testing different than application testing? How is API Monitoring different from simple uptime monitoring? Who is a typical customer of Runscope, what types of challenges and tests are they solving for?

Topic 3 - Walk us thru how the testing works (you mention "no code needed") through the lifecycle of an application. What are some common problems across different platforms (browsers, OS) or different regions of the world?

Topic 4 - API versioning is a major headache. Anything you do to help simplify or manage that for customers? Don’t you still code as a CEO? Do you feel this pain?

Topic 5 - Runscope has a lot of community based projects (link in show notes). How did this come about and what advantages have you seen through the development of an API community?

Topic 6 - With so many APIs these days what's the best way to get started with API testing?

Sunday, April 12, 2015

The Cloudcast #186 - Understanding the Cloud Foundry Foundation

Topic 1 - It’s been nearly two months on the new job. How are things going so far and where haven’t you been speaking - we seen pictures of you everywhere.

Topic 2 - What is Cloud Foundry these days? Sometimes I hear it called “modern middleware”, other times it’s a “platform for modern apps”, or times it’s “advanced container management”.

Topic 3 - Digging into the tech a little bit, Cloud Foundry used to be the platform and then there was BOSH, which was the CF deployment tool. Now there are a bunch of other subset projects, such as Lattice. How does the Foundation manage architectural discussions so this doesn’t turn into OpenStack?

Topic 4 - You’ve been around both open source communities and commercial ecosystems for a while. They’re difference, but similar in ways. Why do you think we’re seeing more projects go towards the Foundation model?

Topic 5 - What are the marketplace goals of the Cloud Foundry Foundation? Where are your boundaries to spread the word vs. moderating messages?

Topic 6 - You’ve built developer communities and ecosystems before. Is there a killer-app “type” or domain that you’re specifically focused on growing or you think will grow faster than others?

Sunday, April 5, 2015

The Cloudcast #185 - Masters, Minions and Pods - Kubernetes 101

Topic 1 - Let’s talk a little bit about your background and why we asked you to come discuss Kubernetes tonight.

Topic 2 - We’re all familiar with Docker at this point, and generally familiar with the underlying container technologies. So where does Kubernetes fit in? (who runs it? what’s the input to the scheduler? what does it use to track resources at the host level? does it assume all machines are the same?)

Topic 2a - What makes Kubernetes easy to use and hard to use?

Topic 2b - Does it use/assume all the native container management tools, or does Kubernetes do some of that tool?

Topic 3 - Let’s walk through the basic concepts and suggested best practices around things like #apps/container, tagging and pods.
Topic 4 - Since Kubernetes came from Google, every just assumes it deals with scale well. But how does the scaling of that control plane work? Is it a single data-center view, multi-data center or smaller segments within a data-center?

Topic 5 - What Google-specific assumptions are built into Kubernetes that might not be broadly applicable to other companies?

Topic 6 - What are some of the common applications that companies use to get started with Kubernetes?

Sunday, March 29, 2015

The Cloudcast #184 - Streaming Analytics for Distributed Applications

Projects of the Week - None this week

Topic 1 - Came out of Stealth recently (3/12), Give a quick overview of the company and the problem you are trying to solve… Given what SignalFx offers, it’s important to understand the people behind it. Let’s start with the background of the team - lots of large, webscale, distributed system background. [how much is “productizing lessons learned”?; how much is “the will be different in 5yrs”?]

Topic 2 - What does streaming analytics mean? Why do companies care about getting analytics faster? Why build an analytics engine to solve a monitoring problem?
Topic 3 - You mention (intro video) that you’re a company that builds services for distributed systems, which are run by product teams, not IT. You were previously at VMware. Can you talk about the different mindset those product teams have vs. IT teams, especially how SignalFx takes their ideas and feedback?

Topic 4 - Walk us through how your customers interact with your service? Where do metrics come from (app, message queue, etc)? How do you secure that API interaction? How are metrics different from logs or events?

Topic 5 - In the same vein as the shift from IT to the Product Groups, your co-founder mentions that Developers are closer to production than ever. What does that mean to the evolution of tools and overall psyche of application developers?
Topic 6 - You mention that SignalFx “double purposes as a Application Intelligence solution”. We’ve been watching lots of interesting SaaS applications  emerge that tend to have a more singular purpose (Logging, PerfMon, AppIntelligence, etc.). Are you hearing from customers that some consolidation of functions is needed?

Saturday, March 21, 2015

The Cloudcast #183 - Container-Centric Application Deployments

Topic 1 - It’s unusual for us to have guests from different companies, but your stories have commonality. But let’s talk about both of your backgrounds (and company backgrounds) first.

Topic 2 - When I was watching this video of Khash (Cloud 66) at this Hacker News meetup in London, it looked to me like a concept I call “unstructured PaaS”, which is sort of a DIY PaaS, with the best-of technologies.

Topic 3 - We’re curious to learn more about ContainerNet, that is the backbone for the container networking of Cloud66 (using Weave technology) and how it really works.

Topic 4 - Both of you are at the forefront of this transition of container-centric application deployments. Where do you see the maturity in the market and what are the next big opportunities?

Topic 5 - You both seem to believe in the model of modularity for these new architectures. Beyond “giving customers choice”, what are the big focus areas in building elements of these modular architectures?

Topic 6 - What are some of the tangible business advantages that you’ve heard from customers when it comes to choice and modularity in this container-centric application model?

Wednesday, February 25, 2015

New ByteSized DevOps Podcasts - Logging, Monitoring and Application State

A few weeks ago we introduced the ByteSized DevOps Podcast series. Initial feedback from the community was very strong, so we've decided to do some more. We plan to release a few every week or two. Let us know what topics you'd like to see covered.

Monitoring and Logging

Stateful vs. Stateless Apps

Sunday, February 15, 2015

Krispy Kreme Challenge 2015

Once again, the best community in technology has come together for an outstanding cause. We want to thank all of our sponsors / donors for helping to raise $4125 for the NC Children's Hospital.

This is the 3rd year in a row that our community has been recognized as the largest donor. We've now raised nearly $15,000 to help children and their families struggling with life threatening diseases.

  • 2013 - $4310
  • 2014 - $5701
  • 2015 - $4125

Aaron and Brian finished the dozen donuts and the five miles is 59m:30s, which is the first time they have completed the challenge in under 1hr! With age comes greater eating skills and superior athletic ability...apparently.


Thursday, February 5, 2015

Introducing "ByteSized" DevOps Podcasts

Today we tried something a little different. As we've shifted the focus of the podcast to have more focus on SaaS, DevOps, Public Cloud and other topics, we've added number of new listeners (up 40% YoY). For many of them, these are new areas of technology. So we thought we'd add something new...

We're calling them "The Cloudcast - ByteSized", and they will be a series of ~ 10min podcasts that just cover the basics of a given topic or technology.

NOTE: We're still going to do (mostly) weekly shows in the normal formal as well. We'll just mix these in from time to time.

Here's the first batch. They should all be consumable independently, but we're also trying to loosely link them together.

You're the best audience in technology, so your feedback is greatly appreciated.