RSS

API Portal News

These are the news items I've curated in my monitoring of the API space that have some relevance to the API definition conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is testing their APIs, going beyond just monitoring and understand the details of each request and response.

The API Portal Outline For A Project I Am Working On

I am working through a project for a client, helping them deliver a portal for their API. As I do with any of my recommendations with my clients, I take my existing API research, and refine it to help craft a strategy to meets their specific needs. Each time I do this it gives me a chance to rethink some of my recommendations I’ve already gathered, as well as learn from new types of projects. I’ve taken the building blocks from my API portal, as well as my API management research, and have taken a crack at organizing them into an outline that I can use to guide my current project.

Here is a walk through of the outline I’m recommending as part of a basic API portal implementation, to support a simple public API:

  • Overview - / - Everything starts with a landing page, with a simple overview of what an API does.

Then you need some basics to help make on-boarding as frictionless as possible, providing everything an API consumer needs to get going:

  • Getting Started - /getting-started/ - Handful of steps with exactly what is needed to get started.
  • Authentication - /authentication/ - An overview of what type of authentication is used.
  • Documentation - /documentation/ - Documentation for the APIs that are available.
  • FAQ - /faq/ - Answer the most common questions.
  • Code - /code/ - Provide code samples, libraries, and SDKs to get going.

Then get API consumers signed up, or able to login and get at their API keys as quickly as you possibly can:

  • Sign Up - /developer/ - Provide a simple sign up for a developer account.
  • Login - /developer/ - Allow users to quickly log back in after they have account.

Next, provide a wealth of communication and support mechanisms, helping make sure developers are aware of what is going on:

  • Blog - /blog/ - A simple blog dedicated to sharing stories about the API.
  • Support - /support/ - Offer up a handful of support channels like email and tickets.
  • Road Map - /road-map/ - Provide a road map of what the future will hold for the API.
  • Issues - /issues/ - Share what the known issues are for the API platform.
  • Change Log - /change-log/ - Publish a log of everything that has changed with the platform.
  • Status - /status/ - Provide a status page showing the availability of API services.
  • Security - /security/ - Publish an overview of what security practices are in place.

Make sure your consumers know how to get involved at the right level, making plans, pricing, and partnership opportunities easy to find:

  • Plans - /plans/ - Offer a single page with API access plans and pricing information.
  • Partners - /partners/ - Share what the partnership opportunities are, as well as existing partners.

Then let’s take care of the legal side of things, making sure API consumers are fully aware of the TOS, and other aspects of operations:

  • Terms of Service - /terms-of-service/ - Make the terms of service easy to find.
  • Privacy - /Privacy/ - Publish a privacy statement for all API consumers.
  • Licensing - /Licensing/ - Share the licensing for API, code, data, and other assets.

I also wanted to make sure I took care of the basics for the developer accounts, flushing out the common building blocks developers will need to be successful:

  • Dashboard - /developer/dashboard/ - Provide a simple, comprehensive dashboard for developers.
  • Account - /developer/account/ - Allow API consumers to change their account information.
  • Applications - /developer/applications/ - Allow API consumers to add multiple applications, and receive API keys for each application.
  • Plans - /developer/plans/ - If there are multiple plans, allow developers to change plans.
  • Usage - /developer/usage/ - Show history of all API usage for API consumers.
  • Billing - /developer/billing/ - Allow API consumers to add, update, and remove billing information.
  • Invoices - /developer/invoices/ - Share a listing, as well as detail of all invoices for use.

Then I had a handful of other looser items that I wanted to make sure were here. Some of these will be for the second phase of developoment, but I want to make sure is on the list:

  • Branding - /branding/ - Providing a logo, branding guidelines, and requirements.
  • Embeddable - /embeddable/ - Share a handful of butts and widgets for use by consumers.
  • Webhooks - /Webhooks/ - Publish details on how to setup and manage webhook notifications.
  • iPaaS - /ipaas/ - Information on Zapier, and other iPaaS integration solutions that are available.
  • Spreadsheets - /spreadsheets/ - Share any spreadsheet plugins or connectors that are available for integration.

That concludes what I’d consider to be the essential building blocks for an API. Sure, there are some more minor APIs that can operate on a bare bones version of this list, but for any API looking to conduct business at scale, I’d recommend considering everything on this list. It reflects what I find across the leading providers like Stripe and Twilio, and reflect what I like to see from an API provider when I am getting up and running using any API.

I have Jekyll templates for each of these building blocks, which use the Bootstrap UI for the layout. I am updating it for this project, then I will publish again as a set of forkable tools that anyone can implement. I’m going to publish a new API portal that runs as an interactive outline of essential building blocks, then creates new Github repository for a user, and publishes each of the selected components to the repo. Providing a buffet of API developer portal tools anyone can put to work for their API without much work.


Operating Your API Portal Using Github

Operating on Github is natural for me, but I am regularly reminded what a foreign concept it is for some of the API providers I’m speaking with. Github is the cheapest, easiest way to launch a public or private developer portal for your API. With the introduction of Github Pages, each Github repository is turned into a place to host any API related project. In my opinion, every API should begin with Github, providing a place to put your API definition, portal, and other elements of your API operations.

If you are just getting going with understand how Github can be used to support your API operations, I wanted to provide a simple checklist of the concepts at play, that will lead you being able to publish your API portal to Github.

  • Github Account - You will need an account to be able to use Github. Anything you do on Github that is public will be free. You can do private portals on Github, but this story is about using it for a public API portal.
  • Github Organization - I recommend starting an organization for your API operations, instead of under just a single users account. Then you can make the definition for the API the first repository, and possibly the portal your second repository you create.
  • Github Repo - A Github repository is basically a folder on the platform which you can start the code, pages, and other content used as part of API operations.
  • Github Pages - Each Github repository has the ability to turn on a public project site, which can be used as a hosting location for a developer portal.
  • Jekyll - Github Pages allows any Github repository to become a website hosting location which you can access via your Github user account, or even provide an address using your own domain.

I recommend every API provider think about hosting their API portal on Github. The learning curve isn’t that significant to get up and running, and if your portal is public, it is free. You can version control, and leverage other key aspects of Github for evolving and managing your API portal. There are a growing number of examples of forkable API portals like from the GSA, or an interesting minimum viable API documentation template from my friend James Higginbotham (@launchany). Demonstrating that the practice is growing, with the number of healthy examples to build on diversifying.

If you need help understanding how to use Github for hosting your API developer portal, feel free to reach out. I am happy to see where I can help. Another thing to note is that this approach to running a Jekyll static website isn’t limited to Github. You can always start the project there, and move off to any Jekyll enabled hosting provider. I run my entire network of websites and API project this way, leveraging Github as my plan A, and AWS as my plan B, with a server image ready to go when I need. Github just provides a number of bells and whistles that make it much more usable, as well as something others can collaborate around, enjoying the network effects that come with using the platform.


A New Minimum Viable Documentation(MVD) Jekyll Template For APIs

I am a big fan of Jekyll, the static content management system (CMS). All of API Evangelist runs as hundreds of little Jekyll driven Github repositories, in a sort of microservices concert, allowing me to orchestrate my research, data, and the stories I tell across all of my projects. I recommend that API providers launch their API portals using Jekyll, whether you choose to run on Github, or anywhere else using the light-weight portable solution. I have several Jekyll templates I use to to fork and turn into new API portals, providing me with a robust toolbox for making APIs more usable.

My friend and collaborator James Higginbotham(@launchany) has launched a new minimum viable documentation (MVD) template for APIs, providing API provides with everything they need out of the gate when it comes to a presence for their API. The MVD solution provides you with a place for your getting started, workflows, code samples, reference material, with OpenAPI as the heartbeat–providing you with everything you need when it comes to API documentation. It all is an open source package available on Github, allowing any API provider to fork and quickly change the content and look and feel to match your needs. Which in my opinion, is the way ALL API documentation solutions should be. None of us should be re-inventing the wheel when it comes to our API portals, there are too many good examples out their to follow.

I know that Jekyll is intimidating for many folks. I’m currently dealing with this on several fronts, but trust me when I say that Jekyll will become one of the most important tools in your API toolbox. It takes a bit to learn the structure of Jekyll, and get over some of the quirks of learning to program using Liquid, but once you do, it will open up a whole new world for you. It is much more than just a static content management system (CMS). For me, it’s most significant strength has become as a data management system (DMS)??, with OpenAPI as the heart. I use Jekyll (and Github) for managing all my OpenAPI definitions, JSON and YAML files, and increasingly publishing my data sets in this way instead of relying on server-side technology. If you are looking for an new solution when it comes to your API portal, I recommend taking a look at what James is up to.


The GSA API Standards With A Working Prototype API And Portal

One way to help API developers understand API design is to provide them with a design guide, helping set a standard for how APIs should be designed across an organization or group. Another way to help developers follow best practices when it comes to API design is to provide them with a working example they can follow when developing their API(s). In my experience people learn API design best practices through following what they know–emulating what they see.

Hang on to that thought, cause now I’m going to blow your mind. Guess how API providers learn how to provide API design guide and working examples? By showcasing working examples of companies, institutions, and government agencies publishing API design guides, working APIs, and portal prototypes. So that other API providers can learn by example! BOOM! Mind blown. :-) An example of this can be found over at the General Service Administration (GSA), with their API standards guide, API prototype, and forkable API portal and documentation–complete with the essential API building blocks.

The GSA’s simple approach to providing a working example of their API standards is refreshing. They have taken an existing GSA data set and launched a prototype API, then they published the API in a complete, working API developer portal with all the essential building blocks of a basic API presence.

  • Getting Started - What you need to get started with the API.
  • Documentation - OpenAPI driven interactive API documentation.
  • Schema - A reference of the fields / schema used for API responses.
  • FAQ - Some basic questions about the API, and more importantly the API prototype.
  • Contact Info - Using Github issues for support, with an accompanying email.

The whole things runs on Github so it is forkable. What is great, is that they also have the source code for the API on Github, essentially providing a working, forkable representation of what is expected in the GSA API design guide. This is how you plant seeds for consistent API design across an organization. An API design guide setting standard, with a working example of what you would like to see as the finished product.

I’m really getting into this approach to defining the API conversation within an organization. I’m working with a handful of large organizations right now to develop API prototypes, evolve my current API portal definition, as well as working to influence the GSA’s approach. If you aren’t familiar with the General Services Administration (GSA), a significant portion of their mission is dedicated to delivering technology services to the over 430 departments, agencies, and sub-agencies in the federal government. The GSA API design guide, prototype, and portal provide a working example other agencies can follow when defining, designing, deploying, and managing their API presence–important stuff.


An Introduction To Github For API Providers

I have had a number of requests from folks lately to write more about Github, and how they can use the social coding platform as part of their API operations. As I work with more companies outside of the startup echo chamber on their API strategies I am encountering more groups that aren't Github fluent and could use some help getting started. It has also been a while since I've thought deeply about how API providers should be using Github so it will allow me to craft some fresh content on the subject.

Github As Your Technical Social Network
Think of Github as a more technical version of Facebook, but instead of the social interactions being centered around wall posts, news links, photos, and videos, it is focused on engagement with repositories. A repository is basically a file folder that you can make public or private, and put anything you want into it. While code is the most common thing put into Github repositories, they often contain data file, presentations, and other content, providing a beneficial way to manage many aspects of API operations.

The Github Basics
When putting Github to use as part of your API operations, start small. Get your profile setup, define your organization, and begin using it to manage documentation or other simple areas of your operations--until you get the hang of it. Set aside any pre-conceived notions about Github being about code, and focus on the handful of services it offers to enable your API operations.

  • Users - Just like other online services, Github has the notion of a user, where you provide a photo, description, and other relevant details about yourself. Avoid making a user accounts for your API, making sure you show the humans involved in API operations. It does make sense to have a testing, or other generic platform Github users accounts, but make sure your API team each have their own user profile, providing a snapshot of everyone involved.  
  • Organizations - You can use Github organizations to group your API operations under a single umbrella. Each organization has a name, logo, and description, and then you can add specific users as collaborators, and build your team under a single organization. Start with a single repository for your entire API operations, then you can consider the additional organization to further organize your efforts such as partner programs, or other aspects of internal API operations.
  • Repositories - A repository is just a folder. You can create a repository, and replicate (check out) a repository using the Github desktop client, and manage its content locally, and commit changes back to Github whenever you are ready. Repositories are designed for collaborative, version controlled engagements, allowing for many people to work together, while still providing centralized governance and control by the designated gatekeeper for whatever project being managed via a repository--the most common usage is for managing open source software.
  • Topics - Recently Github added the ability to label your repositories using what they call topics. Topics are used as part of Github discovery, allowing users to search using common topics, as well as searching for users, organizations, and repositories by keyword. Github Topics is providing another way for developers to find interesting APIs using search, browsing, and Github trends.
  • Gists - A GitHub service for managing code snippets that allow them to be embedded in other websites, documentation -- great for use in blog posts, and communication around API operations.
  • Pages - Use Github Pages for your project websites. It is the quickest way to stand up a web page to host API documentation, code samples, or the entire portal for your API effort.
  • API - Everything on the Github platform is available through the Github API. Making all aspects of your API operations available via an API, which is the way it should be.

Managing API Operations With Github
There are a handful of ways I encourage API providers to consider using Github as part of their operations. I prefer to use Github for all aspects of API operations, but not every organization is ready for that--I encourage you to focus in these areas when you are just getting going:

  • Developer Portal - You can use Github Pages to host your API developer portal--I recommend taking a look at my minimum viable API portal definition to see an example of this in action.
  • Documentation - Whether as part of the entire portal or just as a single repository, it is common for API providers to publish API documentation to Github. Using solutions like ReDoc, it is easy to make your API documentation look good, while also easily keeping them up to date.
  • Code Samples w/ Gists - It is easy to manage all samples for an API using Github Gists, allowing them to be embedded in the documentation, and other communication and storytelling conducted as part of platform operations.
  • Software Development Kits (SDK) Repositories - If you are providing complete SDKs for API integrations in a variety of languages you should be using Github to manage their existence, allowing API consumers to fork and integrate as they need, while also staying in tune with changes.
  • OpenAPI Management - Publish your APIs.json or OpenAPI definition to Github, allowing the YAML or JSON to be versioned, and managed in a collaborate environment where API consumers can fork and integrate into their own operations.
  • Issues - Use Github issues for managing the conversation around integration and operational issues.
  • Road Map - Also use Github Issues to help aggregate, collaborate, and evolve the road map for API operations, encouraging consumers to be involved.
  • Change Log - When anything on the roadmap is achieved flag it for inclusion in the change log, providing a list of changes to the platform that API consumers can use as a reference.

Github is essential to API operations. There is no requirement for Github users to possess developer skills. Many types of users put Github to use in managing the technical aspects of projects to take advantage of the network effect, as well as the version control and collaboration introduced by the social platform. It's common for non-technical folks to be intimidated by Github, ad developers often encourage this, but in reality, Github is as easy to use as any other social network--it just takes some time to get used to and familiar it.

If you have questions about how to use Github, feel free to reach out. I'm happy to focus on specific uses of Github for API operations in more detail. I have numerous examples of how it can be used, I just need to know where I should be focusing next. Remember, there are no stupid questions. I am an advocate for everyone taking advantage of Github and I fully understand that it can be difficult to understand how it works when you are just getting going. 


The Tyk Wordpress API Portal

I am finally seeing more solutions available for API providers when it comes to publishing an portal for their API operations. I've long had my minimum viable API portal definition, which I recently deployed to support the Miami Open211 API, and I also wrote about AWS serverless approach to an API portal. Next up, is a WordPress solution from my API management partner Tyk.

The WordPress API portal solution has the following features:

  • Automatic developer registration on Tyk when developers sign up in WP
  • Configuration of API policies available for token registration
  • Developers may request an access token for the available API policies
  • Automatic or manual approval of key requests
  • Storage of token (references) by name and API policy
  • Revoking of tokens by developer
  • Display usage statistics per key
  • Request quota usage per key

All the basics you need when standing up a basic API operation. If WordPress is your go-to content management system, then it might make sense for you to think about API portal in these terms. The Swiss Federal Railways (SBB-CFF-FFS) is already using the plugin in production, and you can download the plugin on Github, or find it over at WordPress.org.

We need multiple versions of API portals like this, designed for every type of platform a company, organization, institution, and government agency might be already using as a default platform for their operations. This is why API management providers need to follow Tyk's lead and make sure your solution is API driven--practicing what you preach so that your API management solution can be integrated anywhere.


The AWS Serverless API Portal

I was looking through the Github accounts for Amazon Web Services and came across their Serverless API Portal--a pretty functional example of a forkable developer portal for your API, running on a variety of AWS services. It's a pretty interesting implementation because in addition to the tech of your API management it also helps you with the business side of things. 

The AWS Serverless Developer Portal "is a reference implementation for a developer portal application that allows users to register, discover, and subscribe to your API Products (API Gateway Usage Plans), manage their API Keys, and view their usage metrics for your APIs..[]..it also supports subscription/unsubscription through a SaaS product offering through the AWS Marketplace."--providing a pretty compelling API portal solution running on AWS.

There are a couple things I think are pretty noteworthy:

  • Application Backend (/lambdas/backend) - The application backend is a Lambda function built on the aws-serverless-express library. The backend is responsible for login/registration, API subscription/unsubscription, usage metrics, and handling product subscription redirects from AWS Marketplace.
  • Marketplace SaaS Setup Instructions - You can sell your SaaS product through AWS Marketplace and have the developer portal manage the subscription/unsubscription workflows. API Gateway will automatically provide authorization and metering for your product and subscribers will be automatically billed through AWS Marketplace
  • AWS Marketplace SNS Listener Function (Optional) (/listener) - The listener Lambda function will be triggered when customers subscribe or unsubscribe to your product through the AWS Marketplace console. AWS Marketplace will generate a unique SNS Topic where events will be published for your product.

This is the required infrastructure we'll need to get to what I've been talking about for some time with my wholesale API and virtual API stack stories. Amazon is providing you with the infrastructure you need to set up the storefront for your APIs, providing the management layer you will need, including monetization via their marketplace. This is a retail layer, but because your infrastructure is setup in this way, there is no reason you can't sell all or part of your setup to other wholesale customers, using the same AWS marketplace.

I had AWS marketplace on my list of solutions to better understand for some time now, but the AWS Serverless Developer Portal really begins to connect the dots for me. If you can sell access to your API infrastructure using this model, you can also sell your API infrastructure to others using this model. I will have to set up some infrastructure using this approach to better flush out how AWS infrastructure and open templates like this serverless developer portal can help facilitate a more versatile, virtualized, and wholesale API lifecycle. 

There is a more detailed walkthrough of how to get going with the AWS Serverless Developer Portal, helping you think through the details. I am a big fan of these types of templates--forkable Github repositories, with a blueprint you can follow to achieve a specific API deployment, management, or any other lifecycle objective.


A More Distilled Version of An API Getting Started Page On The Home Page Of A Developer Portal

As I look through API portals, profiling the building blocks of successful API platform, I'm always looking for bite-size stories for my readers. I was working to complete my Instagram API definition, as part of my photo API research, and their getting started graphic caught my attention.

This three step getting started visual for Instagram is prominently available on the home page, providing a very distilled down version of what I recommend API providers usually accomplish with a dedicated getting started page.

The featured getting started area provides you with the link to register your application, and overview on authentication and authorization, and a link to all the endpoints, so you can start making requests. I am a big fan of there always being a dedicated getting started page for APIs that I am on-boarding with, but I am always a bigger fan when someone distills these building blocks into something that is way easier to understand, and execute--without all the blah blah blah.

I'm going to consider adding a feature getting started building block, in addition to having the dedicated page. When done right, I think they can accomplsh the same thing, but with one less click--reducing any friction we possibly can in the on-boarding process.


Making Sure The Latest API Pricing Update Is Available In A Developer Portal Messaging Area

I am spending a significant amount of time looking through the pricing pages for leading API providers, working to get a sense for some of the common approaches to API monetization in use across the space. Along the way I am finding some simple approaches from API providers that I wanted to share as bit-size API planning stories here on the blog.

After recently landing on the pricing page for identity service provider Auth0, I saw that they included a simple message and link to the latest blog post about API pricing changes. It is a simple, subtle, yet very prominent approach to making sure your API consumers are aware of pricing changes that may impact their integration. I recently wrote about providing a messaging area to reach developers upon platform login, but this approach from Auth0 is about providing relevant messaging no matter where you are at in the developer portal--not just at login.

Couple of lessons for me, extracted from this approach:

  1. Prominent FMA - Providing a prominently placed, flexible messaging area, can help make sure your API consumers stay up to speed on the latest, when it comes to platform operations. 
  2. Pricing Storytelling - Telling the story around each change in pricing is a critical aspect of operation. Publishing a story on your blog, syndicating out through your social channels, and prominent placement via a FMA is key.

This help rekenforce the fact that we should make sure to regularly communicating out our decisions around API monetization. Also making sure there is a saturation factor to our messaging, and that it is available via our blogs, Twitter, LinkedIn, and any other relevant page or channel used to reach API consumers.

I have a whole list of bit-size API monetization, pricing, and planning stories queued up. I will try to space them out, alongside other stories, but you will just have to suffer a little as I spend time expanding on my API monetization, and API plans research areas. 


Adding a Suggested API Definition for API Portals to My API Management Spec Collection

One layer I am working to add to my API research, are machine readable API definitions that I find, or generate from the APIs of the API service providers I keep an eye on. Within this new layer I'm aggregating these API the specs of the companies who are offering services within the emerging areas of the API sector.

The first area I've started aggregating is within API management. 3Scale was very forward leaning with their willingness to open up their API definitions for their own API management infrastructure APIs. These are API specs that describe all of the features of the 3Scale API management platform, and represent one possible blueprint that API management service providers could emulate.

I have four separate API definitions included from 3Scale on my new page, something that could be broken down further if it makes sense. I also just added a suggested API definition for API portals--crafted by the APIMATIC team. They thought my idea for defining a common set of definitions across the 17+ areas I monitor was an interesting cenough oncept, and was something they wanted to explore further with me.

Zeeshan Ejaz Bhatti of APIMATIC pulled together this spec:

 

 

This is just one potential starting point for providing a common interface for managing your API portal. Zeeshan feels that , if "API-Portals-as-a-Service", and other API management providers agreed on a common publish API format, it would benefit other API service providers like APIMATIC who are providing services to their customers via these API portals.

I agree with Zeeshan, and might play around adding an API facade for managing your API portal using Github + Jekyll, like I do with my standard API Portal template. This is all work in progress. Right now I am just aggregating the API definitions for API service providers that have APIs. In my opinion this layer of the API space will differentiate API service providers from each other, demonstrating which ones will actually scale in the fast growing the API economy.

Next I'm taking API monitoring services like Runscope and API Science, and aggregate their API definitions as part of my research.


Migrating To Version 2.0 Of API Evangelist Portal

It's been a couple years since I started providing public access to my APIs. Historically I haven't really been in the business of generating revenue from my APIs, it is more of a show than anything, but this is something I'm looking to change. This means I need to get my act together when it comes to API design, deployment, management, and monitoring, and practice what I preach.

This is the first step in that direction. I'm launching a new portal to support the handful of APIs I want to make available as part of API Evangelist. This is just a subset of the larger stack of APIs I am working on, but access to my APIs are universal, and one API key will get you access to almost all the APIs across my network. 

I would consider everything you find here BETA, unless stated otherwise. I'm working on way to certify my APIs as ready for production, differentiating the "in development", from the hardened APIs I"m not making changes to. If you have any questions, feel free to submit an issue via this portals Github repository, and I'll do my best to tackle. I'm rapidly evolving these APIs, and others, now that my platform is a little more stable, so come back often.


When Planning Your API Portal Do Not Hide APIs and Always Translate From IT To Something Humans Can Understand

While updating my APIs in higher education research this last week I came across an API portal I think tells an important story about an evolution in how we develop and maintain software. The current wave of web APIs that I cover has been working well because it has exposed a number of valuable resources, that once upon a time were only available in the realm of IT, making these resources accessible to a wider audience of developers, or even business users. 

This evolution has been slowly pulling back the curtain of IT, exposing more of it's inner workings, potentially simplifying, and decoupling valuable digital resources from legacy power structures. An important piece of this transformation is the translation of IT speak to language that the average human can understand, something that is playing out within API developer portals across companies, organizations, institutions, and government agencies. 

I touched on this subject with my work on the Environmental Protection Agency (EPA) Envirofacts API a couple weeks ago. With a little work I was able to expose a rich set of API resources, just by renaming the APIs, and hydrating some of the acronyms that were being used. While this is an example of the intersection of government IT power structures, it represents the same evolution.

As I was updating my university API work, I stumbled across the same evolution occurring at The Ohio State University. When you land on this API developer portal, you are given no signal that this contains course, and other valuable campus related APIs.

This API effort is born out of a classic IT lead project, and while it brings forward some healthy API patterns, it also carries forth the language and culture of IT. I am not saying the team behind the API portal is approaching their portal with intentional bias, these types of efforts are common, and often done by very well meaning groups, who only shortcomings are they don't think about reaching a widest audience possible. 

The idea that anyone should be able to land on an API portal, and be able to understand what is going on, is a foreign concept to many IT and developer groups. They build web service portals, system documentation, and other system doorways, thinking that other like-minded (IT and dev) folks will be coming there. They just aren't used to thinking that business leaders, researchers, data journalists, teachers, students, and any number of other folks might be coming there too.

Most groups that I come across like this, just need to be enlightened, and they will course correct. A smaller, but an often vocal group see no use in anyone outside the IT good ol boy club having access to resources--this is the old IT guard, and how power is protected. They will be gone soon, but for now you will have to just work around them. My goal is to just educate the API portal groups who are willing to listen, and stimulate API efforts from outside of IT lead groups when it is necessary, not to convert the critics. 


Quantifying A Minimum Viable API Footprint Definition As Real APIs.json Driven Portal

I wrote a post the other day laying out what I'd consider a minimum viable footprint for API operations. My vision of just exactly what an API is, has gone beyond just the technical, ever since I started API Evangelist back in July of 2010. Early on I saw this was more than just about the API endpoints, and documentation, code samples, and many other building blocks were essential to the success (or failure) of any API platform, area, or ecosystem.

This recent post was an attempt, here in 2015, to quantify what I would consider to be a minimum definition for API operation. After writing it I wanted to take another stab at actually creating a portal, that would stand up to the API rhetoric that I regularly produce. What better place to start, than my own personal master API stack, where I am working to get control over my own infrastructure. Once I got a version 1.0 definition, I forked it, and setup a default API portal that I am calling my demo APIs.json driven portal.

I have published to Github, a version of my minimum API footprint, which I'm using APIs.json an engine. After looking at 10,000K+ APIs, I have a pretty good idea of what I like to see. This is my interpretation of all that monitoring of the space--a distillation of what I've seen while reviewing APIs. It isn't perfect, but neither is the space that we have, and my portal is just an attempt at quantifying what I'm seeing.

I'm going to create a minimum viable Internet of Things (IoT) version of this portal as well, and use APIs.json to deploy different interpretations of what constitutes a minimum viable API presence. If you have anything you'd like to see in my base template, let me know. If you want to fork and add to, then submit a pull request, even better. I'm just playing around, but also looking to establish a suite of APIs.json driven tools, that help me(and you), better understand the API space.

To help on-board you with my vision, I also added a walk-through for when you land on the site. Something I will be adding to when I have time.


APIs.json Driven Portal

My goal with this portal is to make it all driven by APIs.json, so I can use as a template for portals, that anyone can fork, and quickly generate an API developer that meets my minimum definition of an APIs.json portal.

The navigation, home page, and almost every other section is pulling its content and references from the master APIs.json, as well as the rest of the 25+ APIs in this index.

There are still a lot of manual details for me, when keeping the APIs.json index up to date, but it is paying off. I have parts of my APIs indexed, like I never had before. This is allowing me to keep better in tune with the details of each API.


A Little Standardization Around How We Do Internet of Things (IoT) APIs And Developer Portals Could Go a Long Ways

I was dedicating some time to researching APIs in the Internet of things (IoT) space, and stumbled across the Myfox API, serving the Myfox home security products. While the developer portal for the Myfox API doesn't have everything I'd like to see in an IoT developer portal, it is one of the better one's I've seen.

If you think web or mobile app developers suck at providing a simple, and a coherent developer portal and experience, device platform providers are even worse. The Myfox API, has the essential building building blocks, with a simple overview, authentication info, simple Swagger driven API documentation, application management tools, and a clear terms of service for the platform.

Additionally, there is a clear link between the primary Myfox website, the developer portal, and a myfox.me user portal, for device consumers and would-be developers to easily on-board with everything. I haven't fully profiled that Myfox platform, but at first glance it is clean, well laid out, and most importantly the link between the website, API, and user accounts is clear, and easy to navigate.

As I think this through, and after recently finishing a white paper called Building an API-driven Ecosystem for the Internet of Things, for the now defunct GigaOm, I can't help but think the IoT space is going to need to agree on some sort of standard approach for not just managing how they design, deploy, and manage APIs, but also how they deliver developer and user portals. To help you visualize what I mean, consider the number of Internet connected devices any single household will own, all with their own portal, and on-boarding process. #FUN

In short, developers, and end-users will get IoT account fatigue pretty quick. I don't think that we are going to standardize everything across the space, I am not delusional, but I can't help but think that the sharing of some common blueprints, could go a long ways in reducing friction for developers, and device consumers. I'll take my research from the GigaOm white paper and produce a blueprint, that IoT device makers can use, to to hopefully be more successful in deploying their developer portals, platform accounts, and maybe help influence a tiny bit of consistency in how IoT platforms function.


I Wish API Providers Published Their Developer Portals On Github So I Could Submit Pull Requests

I spend a lot of time looking through the developer portals of API providers. I see a lot of things, both the good and bad, while navigating these portals, and while some of the bad stuff I see are way too big for me to doing anything, there are many little things I see that I could help do something about. Sometimes it is just spelling mistakes, sometimes broken links, and other times I want to rewrite API descriptions, and add to the resources that are available for an API.

During these travels, I got to thinking...what if API providers published their developer portals to Github? Just like I do with my API Evangelist sites. Then I could fork their portal, and submit pull requests with rewrites of the description of what your API does, adding little tweaks, here and there, helping polish the portal--potentially making things easier for developers. I’m sure this thought scares the hell out of some companies, who can’t imagine, allowing outside input like this, but if you sit down and think about it for a while, it just might make sense.

When I was in Washington D.C., I created a default portal template that runs on Github. I might upgrade this, and make it available for API providers to use as a forkable template, to seed any API developer portal. Doing this makes it easy to provide a base set of building blocks that every API provider should start with, allowing them to delete the elements that they don’t want. I am also considering making a default one, that can be used as a starter developer portal for Internet of Things (IoT) platforms.

I thoroughly enjoy the pull requests that I get for spelling and grammar mistakes I made on API Evangelist, from some of my favorite grammar trolls. Running all of my websites on Github Pages has changed the way I tell stories, and interact with my audience. I’ve always encouraged API providers to use Github for an increasing number of API management tasks, but actually running your entire portal as a Github repo, would this to a new level. This approach could be used for larger, or smaller developer platforms, and I even use Github pages as a front-end for private API developer portals--ask me how.

It is just a suggestion for you API providers, and soon to be API providers. Github provides a very low cost, scalable way to launch your portal, and you never know, people like me who spend a lot of time in your developer portal, might actually help make it a better place. I wish more API providers would publish their portal as a Github repository, as well as graphic designers in the space help craft template portals, that providers can put to work. My portal definitions will always reflect best practices that I see across the API landscape, but they won’t always be the slickest looking when it comes to graphic design—just not my core competency.


Why Would Want To List All Your Universities Web Services (APIs) Out In The Open, Via Central Portal? What A Security Risk!

I went up to California State University Channel Islands the other day to talk APIs with their tech team, and I was happy to find at least one strong API skeptic on the team. API skeptics also give me material for stories, so I thoroughly enjoy coming across them, and telling these stories is how keep polishing my argument for the next API skeptic encounter at campus IT, at the higher educational institutions that I visit.

During the discussion I was asked several interesting questions, the first one was: why would you want to list of your web services (APIs) for your public university, out in the open, via a central portal?? What a security risk!!

Sorry, But Security Through Obscurity Is Not A Strategy
I’m sorry, hiding things, and hoping nobody finds them is not a valid IT strategy. You are a public university, and you should have a sophisticated identity and access management, as well as well tight approach to security. If you can’t properly secure an API resource available at a public URL, you shouldn’t be running campus IT, sorry. You should be able to provide a public description, of what resources are available, without giving away the farm.

The Feeling You Have —It Comes From Legacy Power And Control
As an IT professional, if you don’t acknowledge that power and control exists within classic IT processes, you are in denial. I’ve taken over numerous company IT operations, and there is ALWAYS political struggles between IT and the rest of operations. If you list all digital resources in a single portal, and make them available in a self-service way, it disrupts existing power and control structures. When you make things hard to find, you play a central role in people finding these resources, and you are in power—adding to decades of legacy stories about IT being a negative force in operations. Self-service access to campus resources is the future, no matter how much you resist.

Centralized Location To Aggregate All Digital Resources
A centralized API portal provides a single place for anyone to discover or share their digital resources. Like the main website, administrators, faculty, students, parents, and the public will know there is a single place to find machine readable versions of campus resources. API portals are more than just a listing of APIs, it includes documentation, code samples, widgets, buttons, spreadsheet connectors, visualizations, blogs, twitter accounts, and other resources that can turn a simple portal into an active ecosystem where everyone collaborates around campus resources.

Self-Service Access For 3rd Party Vendors Delivering Vital Campus Services
It takes resources to engage with 3rd party vendors on campus, and a single, self-service portal provides a standardized way for vendors to access the institutional resources they need to deliver the services they are bringing to campus operations. There is no reason that IT or departmental contacts should ever be bottlenecks in delivering the information vendors will need. API portals should provide them with the access they need, along with proper identity and access management, and monitoring of exactly how vendors are accessing, and putting campus resources to use (or not).

Interoperability Between Higher Educational Institutions
Most higher educational institutions are part of a larger network of institutions, and at the very least have relationships with other schools, in which information, and resources are shared. As with vendors, much of this can be provided via a self-service API portal, where institutions can find the data, content, and other resources they need, able to access only the resources they are supposed to during the fullfillment of established relationships.

Smoother Interactions With Local, State, And Federal Government
Higher educational institutions have regular integration points with local, state, and federal government agencies. Data is reported on a regular basis, back and forth, and a central API portal is excellent for aggregating the commons resources a government agency will be asking for. API portals are not just for student hackers, or 3rd party developers, they are increasingly how government agencies are sharing, and getting what they need to regulate, monitor industry, and govern more effectively.

It Is About People Getting Access To What They Need
Central API portals are about making sure people get access to what they need, and at a public university, this should be priority number one. You have data, content, and other digital resource students, faculty, administrators, vendors, government, parents and alumni need, and because you aren’t confident in securing it, from a handful of hackers you are going to hide this away, and forgo all the benefits? I’m sorry, the perceived negatives just don’t out-weighs the positives. There are proven ways to secure APIs, and for much of university operations, it won’t hurt to provide a title, description, and location of a resource, and let identity and access management handle who should be able to get at resources—or not.

Bringing IT Out of Shadows Across Campus Operations
Security is a concern, and should be front-and-center in all conversations, but it is too often used to hide away insecurities, incompetency, and shortcomings in security, rather than actual address the root of security concerns. If a resource is accessible via Internet protocols, over campus networks, it should be mapped out, with the definition made publicly available in a sensible way. This conversation should involve careful consideration what resource should be public, and what should remain private, with a heavy emphasis on transparency at a public university.

Between 2000 and 2007, we start seeing a shift of IT services into the cloud, and much of this has happened, as un-sanctioned shadow IT, by administrators, faculty, and students looking to get their work done, because IT hasn’t been able to keep up. The cloud evolution of IT has been all API driven, and making IT resources available to the public at Amazon sounded insane at first, but it has changed not just how Amazon operates, but has evolved the entire world deploys software architecture—using APIs.

There are many great examples of leading companies, making personally identifiable information, banking, healthcare, and other vital data and content available on the open Internet, in a secure way. It is not a security risk to share information, and provide access to your valuable resources, in a central, publicly available portal--if you do it right. Who knows, if you do open up, there may be some unintended consequences, leading to IT being seen as positive influence on the education process, as opposed to the roadblock image it often has on campuses across the country.


New API Management Providers: Clean, Modern API Portals With ReadMe

It makes me happy to see new arrivals in the world of API management service providers, especially after all the consolidation we saw last year with many of the 1st and 2nd wave of providers like Mashery, Vordel, Layer7, and Apiphany. One of the new API management providers that have emerged is Readme, who is looking to provide an attractive, simple, and intuitive way to launch developer portals for your APIs.

Readme reminds me of some of the landing page tools designed for web marketers over the last decade, to provide informative gateways for site visitors, but ReadMe is all about providing meaningful doorways to our API resources. First, I like ReadMe's definition of what is a developer hub?

  • Documentation - Topical guides, tutorials and troubleshooting.
  • API Reference - Low-level, deep-dive reference material.
  • Community - Provide support and answer questions.

Their definition of what is a developer hub is goes beyond just the technical documentation as what many developers see, and acknowledges that your portal should be about providing the resources necessary to educate API consumers, and potentially build community around their needs. ReadMe provides the essential building blocks I'd expect of any modern API portal builder:

  • Theme Builder - Easily create a beautiful dev community that matches your brand.
  • Editor - Markdown-based drag-and-drop editor makes documentation almost fun.
  • API Explorer - Let users play with your API right inside the documentation.
  • Application Keys - Your users can view their application keys embedded right in the docs.
  • Support - Let users ask questions and request features in the support forums.

Then ReadMe goes a little further to include some concepts that I see in some of the leading API providers, features that go beyond just basic features, allowing you to better manage your API portal:

  • Collaboration - Crowdsource your docs! Users can keep docs current by suggesting changes.
  • GitHub Sync - Keep auto-generated reference docs synced with your actual code changes.
  • Versioning - Maintaining old or testing beta versions of your docs is a breeze.

In 2014, emulating the social elements that Github has introduced into the world of coding, in your API program is essential. You cannot manage your entire API community by yourself, and including your developer in the process is essential. This adds relevant layers to the term "open" that everyone likes to use, providing the roots you will need to actually build trust with your developers, in something that goes both ways, and will also grow your own trust of developed within your own API community.

I'm keeping an eye on what ReadMe is up to, alongside the other API management providers I've been tracking on. I haven't give a lot of attention to the API management space in the last year, as I've been focusing on the faster growing areas like API discovery, design, and integration, but now that I see new players stepping up, I will make sure and give the area equal attention in my research and monitoring.


Coca-Cola to Launch New Developer Portal and APIs Site


Coca-Cola will launch a new developer site, powered by Apigee, to simplify the process of accessing and using the company's internal and public-facing APIs. Coca-Cola is just one of the many enterprises leveraging APIs for marketing, operations, mobile app development and more. In June, ProgrammableWeb reported that Oreo is using APIs to build and promote a groundbreaking digital marketing and social campaign in China. Most of the APIs used by Coca-Cola are internal and not public facing. However, successful APIs are having an impact on the company's entire operations. Kevin Flowers, CTO of Coca-Cola Enterprises, spoke at the Business of APIs Conference in December 2013. In a presentation at the conference, he said: Our [Coca-Cola's] entire operations are now impacted by successful APIs. We're an operations company. We run distribution, manufacturing, sales — so those operations, everything from procurement to our customer engagement programs, or supply chain, everything now has APIs enabling it. And I think that's a really important point because we're not a dot-com. Internal production APIs used by Coca-Cola include Foursquare, Amazon S3, Twilio, SmartPay, Salesforce Chatter. There is also a Coca Cola Enterprises API powered by Mashery that allows the company's customers, suppliers, consumers and partners to build innovative CCE Web and mobile applications. Coca-Cola is also using APIs to create innovative IoT-to-business applications. Nordic APIs reported in April that Coca-Cola had programmed vending machines to perform self-diagnostics. When the program determines that repairs are needed, the vending machine utilizes the Twilio API to send a text to repair contractors requesting that the machine be serviced. According to Computerworld UK, Coca-Cola started using APIs earlier this year to build and promote a Coke rewards consumer-loyalty-type marketing strategy. When it comes to public-facing APIs, Flowers said at the Business of APIs Conference: From a public-facing API strategy, we really are not expecting our corporation [Cola-Cola] necessarily to be one that there's a large developer community just hacking and growing and building applications on our data. Where we see the public facing is we're able to leverage the APIs to an extended developer community. So for us it would be more going to mobile developers or cloud-based platforms to say, "Look, here's the data. Let's work together and innovate" versus a developer community. Find out more about the existing Coca-Cola Enterprises API at the official Coke CCE website.

URL: http://apiscene.com/out?url=http%3A%2F%2Fwww.programmableweb.com%2Fnews%2Fcoca-cola-to-launch-new-developer-portal-and-apis-site%2Fbrief%2F2014%2F09%2F18

Continuing With The API Restaurant Analogy

I began exploring the use a restaurant menu to help people understand API copyright, and how your API definition is not your secret sauce, and that there is so much more to your API that just the surface area.

I'm continuing my exploration of using restaurants as an analogy to help onboard people with not just API copyright, but introducing APIs to the masses--helping them understand the layers to the API space, in a way that is familiar.

To prepare for future stories and conversations I'm expanding on my API definition = Menu analogy with:

  • API Providers = Restaurant Owners
  • Developer Portal = Restaurant
  • APIs = Food & Drink Items
  • API Definition + Docs = Menu
  • API Consumers (Apps) = Restaurant  Customers (People)
  • Freemium - Samples Out Front
  • Partner Access = Catering
  • Terms of Service = We Reserve The Right To Refuse - No Substitutions
  • API Service Providers = Restaurant Equipment Providers

Obviously it is work in progress. I will need better stories and visuals to get the concept across. Then once I start working through the different types of restaurants (ie. American, Chinese, Mexican, Fusion, etc), I'm sure I'll flush out more, while also potentially hitting some dead ends.

Explaining what an API is, let alone the versatility and flexibility of an API, is really hard. I need as many tools in my API toolbox to help get people up to speed, understanding what an API is, and how it can apply to their business. We'll see where I go with the restaurant analogy, who knows it might be a dead end. It proved to be extremeley valuable in helping people understand API copyright, and I'm hopefull it will do the same for APIs in general.

Let me know your thoughts...


Chief Data Officer Needs To Make The Department Of Commerce Developer Portal The Center Of API Economy

Today, the U.S. Secretary of Commerce Penny Pritzker (@PennyPritzker), announced that the Department of Commerce will hire its first-ever Chief Data Officer. I wanted to make sure that when this new, and extremely important individual assumes their role, they have my latest thoughts on how to make the Department of Commerce developer portal the best it possibly can be, because this port will be the driving force behind the rapidly expanding API driven economy.

Secretary Pritzker does a pretty good job of summing up the scope of resources that are available at Commerce:

Secretary Pritzker described how the Department of Commerce’s data collection – which literally reaches from the depths of the ocean to the surface of the sun – not only informs trillions of dollars of private and public investments each year and plants the seeds of economic growth, but also saves lives.

I think she also does a fine job of describing the urgency behind making sure Commerce resources are available:

Because of Commerce Department data, Secretary Pritzker explained, communities vulnerable to tornadoes have seen warning times triple and tornado warning accuracy double over the past 25 years, giving residents greater time to search for shelter in the event of an emergency.

To understand the importance of content, data and other resources that are coming out the Department of Commerce, you just have to look at the list of agencies that are underneath Commerce, who already have API initiatives:

Then take a look at the other half, who have not launched APIs:

The data and other resources available through these agencies, underneath the Department of Commerce, reflect the heart of not just the U.S. economy, but the global economy, which is rapidly being driven by APIs, which are powering stock markets, finance, payment providers, cloud computing, and many other cornerstones of our increasingly online economy.

Look through those 13 agencies, the resource they manage are all vital to all aspects of the economy telecommunications, patents, weather, oceans, census, to other areas that have a direct influence on how markets work, or don't work.

I’m all behind the Department of Commerce, hiring a Chief Data Officer (CDO), but my first question is, what will this person do? 

This leader, Secretary Pritzker explained, will oversee improvements to data collection and dissemination in order to ensure that Commerce’s data programs are coordinated, comprehensive, and strategic.

Yes! I can get behind this. In my opinion, in order for the new CDO to do this, they will have to quickly bring all of the agencies /developer program up to a modern level of operation. There is a lot of work to be done, so let's get to work exploring what needs to happen. 

A Central Department of Commerce Developer Portal To Rule Them All
Right now the Department of Commerce developer portal at commerce.gov/developer, is just a landing page. An after thought, to help you find some APIs--not a portal. The new CDO needs to establish this real estate as the one true portal, which provides the resources other agencies will need for success, while also providing a modern, leading location for developers of web, mobile, Internet of things applications, and data journalists, or analyst to come to find the data they need. If you need a reference point, go look at Amazon Web Services, SalesForce, eBay or Googe’s developers areas—you should see this type of activity at commerce.gov/developer.

Each Agency Must Have Own Kick Ass Developer Portal
Following patterns set forth by their parent, Department of Commerce portal, each agency underneath needs to posses their own, best of breed developer portal, providing the data, APIs, code, and other resources, that public and private sector consumers will need. I just finished looking through all the available developer portals for commerce agencies, and their is no consistency between them in user experience (UX), API design, or resources available. The new CDO will have to immediately get to work on taking existing patterns from the private sector, as well as what has been developed by 18F, and set a establish common patterns that other agencies can follow when designing, developing and managing their own agencies developer portal.

High Quality, Machine Readable Open Data By Default
The new CDO needs to quickly build on existing data inventory efforts that has been going on at Commerce, making sure any existing projects, are producing machine readable data by default, making sure all data inventory is available within their agency's portal, as well as at data.gov. This will not be a one time effort, the new CDO needs to make sure all program and project managers, also get the data steward training they will need, to ensure that all future work at the Department of Commerce, associated agencies, and private sector partners produces high quality, machine readable data by default.

Open Source Tooling To Support The Public And Private Sector
Within each of the Commerce, and associate agency developer portals, there needs to be a wealth of open source code samples, libraries and SDKs for working with data and APIs. This open source philosophy, also needs to be applied to any web or mobile applications, analysis or visualization that are part of Commerce funded projects and programs, whether they are from the public or private sector. All software developed around Commerce data, and receive public fundeing should be open source by default, allowing the rest of the developer ecosystem, and ultimately the wider economy to benefit and build on top of existing work.

Machine Readable API Definitions For All Resources
This is an area, that is a little bit leading edge, even for the private sector, but is rapidly emerging to play a central role in how APIs are designed, deployed, managed, discovered, tested, monitored, and ultimately integrated into other systems and applications. Machine readable API definitions are being used as a sort of central truth, defining how and what an API does, in a machine readable, but common format, that any developer, and potentially other system can understand. Commerce needs to ensure that all existing, as well as future APIs developed around Commerce data, possess a machine readable API definition, which will allow for all data resources to be plug and play in the API economy.

Established An Assortment Of Blueprints For Other Agencies To Follow
The new Commerce CDO will have to be extremely efficient at establishing successful patterns that other agencies, projects and programs can follow. This starts with developer portal blueprints they can follow when designing, deploying and managing their own developer programs, but should not stop there, and Commerce will need a wealth of blueprints for open source software, APIs, system connectors, and much, much more. Establishing common blueprints, and sharing these widely across government will be critical for consistency and interoperability--reducing the chances that agencies, or private sector partners will be re-inventing the wheel, while also reducing development costs.

Establish Trusted Partner Access For Public And Private Sector
Open data and APIs, do not always mean publicly available by default. Private sector API leaders have developed trusted partner layers to their open data and API developer ecosystems, allowing for select, trusted partners greater access to resources. An existing model for this in the federal government, is within the IRS modernized e-file ecosystem, and the trusted relationships they have with private sector tax preparation partners like H&R Block or Jackson Hewitt. Trusted partners will be critical in Department of Commerce operations, acting as private sector connectors to the API economy, enabling higher levels of access from the private sector, but in a secure and controlled way that protects the public interest.

Army of Domain Expert Evangelists Providing A Human Face
As the name says, Commerce spans all business sectors, and to properly "oversee improvements to data collection and dissemination in order to ensure that Commerce’s data programs are coordinated, comprehensive, and strategic”, the CDO will need another human layer to help increase awareness of Commerce data and APIs, while also supporting existing partners and integrators. An army of evangelists will be needed, possessing some extremely important domain expertise, across all business sectors, that Department of Commerce data and resources will touch. Evangelism is the essential human variable, that makes the whole open data and API algorithm work, the new CDO needs to get to work writing a job description, and hiring for this army—you will need an 18F, but one that is dedicated to the Department of Commerce.

Department of Commerce As The Portal At The Center Of The API Economy
The establishment of CDO at the Department of Commerce is very serious business, and is a role that will be central to how the global economy evolves in the coming years. The content, data, and digital resources that should, and will be made available at commerce.gov/developer and associated agencies, will be central to the health of the API driven economy.

Think of what major seaports have done for the the economy over the last 1000 years, and what role Wall Street has played in the economy over the last century—this is the scope of the commerce.gov/developer portal, which is ultimately the responsibility of the new Department of Commerce Chief Data Officer.

When the new CDO get started at the Department of Commerce, I hope they reach out to 18F, who will have much of what you need to get going. Then sit down, read this post, as well my other one on, An API strategy for the U.S. government, and once you get going, if you need any help, just let me know—as my readers know, I’m full of a lot of ideas on APIs.


Low Hanging Fruit For API Discovery In The Federal Government

I looked through 77 of the developer areas for federal agencies, resulting in reviewing approximately 190 APIs. While the presentation of 95% of the federal government developer portals are crap, it makes me happy that about 120 of the 190 APIs (over 60%) are actually consumable web APIs, that didn't make me hold my nose and run out of the API area. 

Of the 190, only 13 actually made me happy for one reason or another:

Don't get me wrong, there are other nice implementations in there. I like the simplicity and consistency in APIs coming out of GSA, SBA, but overall federal APIs reflect what I see a lot in the private sector, some developer making a decent API, but their follow-through and launch severeley lacks what it takes to make the API successful. People wonder why nobody uses their APIs? hmmmmm....

A little minimalist simplicity in a developer portal, simple explanation of what an API does, interactive documentation w/ Swagger, code libraries, terms of service (TOS), wouild go a looooooooooooong way in making sure these government resources were found, and put to use. 

Ok, so where the hell do I start? Let's look through theses 123 APIs and see where the real low hanging fruit for demonstrating the potential of APIs.json, when it comes to API discovery in the federal government.

Let's start again with the White House (http://www.whitehouse.gov/developers):

Only one API made it out of the USDA:

Department of Commerce (http://www.commerce.gov/developer):

  • Census Bureau API - http://www.census.gov/developers/ - Yes, a real developer area with supporting building blocks. (Update, News,( App Gallery, Forum, Mailing List). Really could use interactive document though. There are urls, but not active calls. Would be way easier if you could play with data, before committing. (B)
  • Severe Weather Data Inventory - http://www.ncdc.noaa.gov/swdiws/ - Fairly basic interface, wouldn’t take much to turn into modern web API. Right now its just a text file, with a spec style documentation explaining what to do. Looks high value. (B)
  • National Climatic Data Center Climate Data Online Web Services - http://www.ncdc.noaa.gov/cdo-web/webservices/v2Oh yeah, now we are talking. That is an API. No interactive docs, but nice clean ones, and would be some work, but could be done. (A)
  • Environmental Research Division's Data Access Program - http://coastwatch.pfeg.noaa.gov/erddap/rest.html - Looks like a decent web API. Wouldn’t be too much to generate a machine readable definition and make into a better API area. (B)
  • Space Physics Interactive Data Resource Web Services - http://spidr.ngdc.noaa.gov/spidr/docs/SPIDR.REST.WSGuide.en.pdf - Well its a PDF, but looks like a decent web API. It would be some work but could turn into a decide API with Swagger specs. (B)
  • Center for Operational Oceanographic Products and Services - http://tidesandcurrents.noaa.gov/api/ - Fairly straightforward API, Simple. Wouldn’t be hard to generate interactive docs for it. Spec needed. (B)

Arlington Cemetary:

Department of Education:

  • Department of Education - http://www.ed.gov/developers - Lots of high value datasets. Says API, but is JSON file. Wouldn’t be hard to generate APIs for it all and make machine readable definitions. (B)

Energy:

  • Energy Information Administration - http://www.eia.gov/developer/ - Nice web API, simple clean presentation. Needs interactive docs. (B)
  • National Renewable Energy Laboratory - http://developer.nrel.gov/ - Close to a modern Developer area with web APIs. Uses standardized access (umbrella). Some of them have Swagger specs, the rest would be easy to create. (A)
  • Office of Scientific and Technical Information - http://www.osti.gov/XMLServices - Interfaces are pretty well designed, and Swagger specs would be straightforward. But docs are all PDF currently. (B)

Department of Health and Human Services (http://www.hhs.gov/developer):

Food and Drug Administration (http://open.fda.gov):

Department of Homeland Security (http://www.dhs.gov/developer):

Two losse cannons:

 Department of Interior (http://www.doi.gov/developer):

Department of Justice (http://www.justice.gov/developer):

Labor:

  • Department of Labor - http://developer.dol.gov/ - I love their developer area. They have a great API, easy to generate API definitions. (A)
  • Bureau of Labor Statistics - http://www.bls.gov/developers/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)

Department of State (http://www.state.gov/developer):

Department of Transportation (http://www.dot.gov/developer):

Department of the Treasury (http://www.treasury.gov/developer):

Veterans Affairs (http://www.va.gov/developer):

Consumer Finance Protectection Bureau:

Federal Communications Commission (http://www.fcc.gov/developers):

Lone bank:

  • Federal Reserve Bank of St. Louis - http://api.stlouisfed.org/ - Good API and area, would be easy to generate API definitions. (B)

General Services Administration (http://www.gsa.gov/developers/):

National Aeronautics and Space Administration http://open.nasa.gov/developer:

Couple more loose cannons:

Recovery Accountability and Transparency Board (http://www.recovery.gov/arra/FAQ/Developer/Pages/default.aspx):

Small Business Administration (http://www.sba.gov/about-sba/sba_performance/sba_data_store/web_service_api):

Last but not least.

That is a lot of potentially valuable API resource to consume. From my perspective, I think that what has come out of GSA, SBA, and White House Petition API, represent probably the simplest, most consistent, and high value targets for me. Next maybe the wealth of APis out of Interior and FDA. AFter that I'll cherry pick from the list, and see which are easiest. 

I'm lookig to create a Swagger definition for each these APIs, and publish as a Github repository, allowing people to play with the API. If I have to, I'll create a proxy for each one, because CORS is not common across the federal government. I'm hoping to not spend to much time on proxies, because once I get in there I always want to improve the interface, and evolve a facade for each API, and I don't have that much time on my hands.


Looking At 77 Federal Government API Developer Portals And 190 APIs

I spent most of the day yesterday, looking through 77 of the developer portals listed on the 18F Github portal. While I wanted to evaluate the quality and approach of each of the agencies, my goal for this review cycle was to look for any APIs that already had machine readable API definitions, or would be low hanging fruit for the creation of Swagger definitions, as part of my wider API discovery work.

I had just finished updating all my API Evangelist Network APIs to use verion 0.14 of APIs.json, and while I wait for the search engine APIs.io to update to support the new version, I wanted to see if I could start the hard work of applying API discovery to federal government APIs. 

Ideally all federal agencies would publish APIs.json on their own, placing it within the root of their domain, like they do with data.json, and provide an index of all of their APIs. Being all to familiar with how this stuff work, I know that if I want this to happen, I will have to generate APIs.json for many federal agencies first. However for the APis.json to have their intended impact, I need many of the APIs to have machine readable API definitions that I can point to--which equals more work for me! yay? ;-(

My thinking is that I will look through all of the 77 developer areas, and resulting APIs looking for the low hanging fruit. Basically I would grade each API on its viability to be included in my federal government API discovery work. I spent minimal amount of time look at each API, and in some cases looking for the API, before giving up. I would inspect the supporting developer area, and the actual interface for complexity, helping me understand how hard it would be to hand craft a Swagger spec, and APIs.json for each agency and their APIs. 

(warning contains my raw un-edited notes from doing this research, not suitable for children)

As I went through, I wrote a couple of notes:

  • National Climate Data Center is nice looking, and is a high profile--they should have a kick ass API!
  • Inversely NOAAA Climate Data Online very simple, clean and well done.
  • Department of Education is acceptable as dev area, only because of Socrata
  • National Renewable Energy Laboratory just gets it. They just get it.
  • Some of these are billed as developer areas, but really just a single API. It is a start I guess. 
  • I love me some Department of Labor. Their developer area and API is freak'n cool!
  • MyUSA citizen API has oAuth!!! WTF. How did I not notice this before? Another story, and work here.
  • MyUSA has really good, simple, high value, API resources. 
  • NASA ExoAPI not just API cool, but space cool!!
  • FOIA needs a fucking API. No excuses. FOIA needs an API!
  • Some APIs it might be better to go back to data source and recreate API from scratch, they are so bad.

I wanted to share some of my notes, before the long list of developer areas, and their APIs. There are some specific notes for each APIs, but much of it is very general, helping grade each API, so I can go back through the list of B grade or higher APIs, and figure out which are candidates for me to create a Swagger API definition, APIs.json and ultimately adding to APIs.io. 

For this work I went down the 77 federal agency links, which were billed as developer areas, but many were single APIs. So when a developer area resulted in multiple APIs, I grouped them together, and many of the agencies who have a single API I will group together, and include my commentary as necessary. I'm leaving the URLs visible to help as a reference, show the craziness of some of them, and because it would have been sooooo much work to apply all of them.

Let's start with the White House(http://www.whitehouse.gov/developers):

Next up is the USDA (http://www.usda.gov/wps/portal/usda/usdahome?navid=USDA_DEVELOPER), which is a hodgepodge of service, no consistency whatsoever between services in interface, supporting content or anything. 

Overall I give USDA a D on all their APIs. A couple might be high value sources, and going after, but definitely not low hanging fruit for me. It would be easier to tackle as independent project for generating brand new APIs.

Next up is Department of Commerce (http://www.commerce.gov/developer), who definitely high some higher value resources, as well as some health API initiatives. 

Next with the Department of Defense (http://www.defense.gov/developer/)There are 8 things billed as aPIs, with a variety of datasets, API like things, and web services available.  Not really sure whats up? (D)

We have one by itself here:

Then the department of education who is just riding their data.gov work as API area:

  • Department of Education - http://www.ed.gov/developers - Lots of high value datasets. Says API, but is JSON file. Wouldn’t be hard to generate APIs for it all and make machine readable definitions. (B)

Next some energy related efforts:

  • Department of Energy - http://www.energy.gov/developers - Lots of datasets. Pretty presentation.  Could use some simple APIs,. Wouldn’t be much to pick high value data sets and create cache of them. (C)
  • Energy Information Administration - http://www.eia.gov/developer/ - Nice web API, simple clean presentation. Needs interactive docs. (B)
  • National Renewable Energy Laboratory - http://developer.nrel.gov/ - Close to a modern Developer area with web APIs. Uses standardized access (umbrella). Some of them have Swagger specs, the rest would be easy to create. (A)
  • Office of Scientific and Technical Information - http://www.osti.gov/XMLServices - Interfaces are pretty well designed, and Swagger specs would be straightforward. But docs are all PDF currently. (B)

Moving on to the Department of Health and Human Services (http://www.hhs.gov/developer), which all of their APis are somewhat cosistent, and provide simple resources:

The Food and Drug Administration (http://open.fda.gov) is one of the agencies that is definitely getting on board with APIs, they have some pretty nice implementations, but there there are some not so nice ones that need a lot of work:

Next up the Department of Homeland Security (http://www.dhs.gov/developer), where they have three APIs (its a start):

Then we have two agencies that have pretty simple API operations, so I'll group together:

Then we have several API developer efforts under the Department of Interior (http://www.doi.gov/developer):

Now we have some APIS coming out of law enforcement side of government, starting with Department of Justice (http://www.justice.gov/developer):

Now we get to one of my favorite afforts in the federal government

  • Department of Labor - http://developer.dol.gov/ - I love their developer area. They have a great API, easy to generate API definitions. (A)
  • Bureau of Labor Statistics - http://www.bls.gov/developers/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)

Next we have the API efforts from Department of State (http://www.state.gov/developer):

Moving on to the Department of Transportation (http://www.dot.gov/developer):

Now let's head over to the Department of the Treasury (http://www.treasury.gov/developer):

The Department of Veterans Affairs (http://www.va.gov/developer) has some hope, because of the work I did in the fall.

Moving to another on of my favorite agencies, well quasi gov agencies:

One agency that appears to be on radar, but I really can't tell what is going on API wise:

  • Environmental Protection Agency - http://www.epa.gov/developer/ - There is a really nice layout to the area, with seemingly a lot of APIs, and they look like web APIs, but it looks like one API being represented as a much of methods? Would be too much work, but still hard to figure out WTF. (C)

The the Federal Communications Commission (http://www.fcc.gov/developers) has a lot of APIs going on, in various states of operation:

All by itself on the list, we have a bank one lonely bank:

  • Federal Reserve Bank of St. Louis - http://api.stlouisfed.org/ - Good API and area, would be easy to generate API definitions. (B)

The General Services Administration (http://www.gsa.gov/developers/) definitely is ahead of the game when it comes to API design and deployment:

Once I reached the National Aeronautics and Space Administration http://open.nasa.gov/developer I found some really, really cool APIs:

Grouping a couple loose agencies together:

The Recovery Accountability and Transparency Board (http://www.recovery.gov/arra/FAQ/Developer/Pages/default.aspx) has some APIs to look at: 

The Small Business Administration (http://www.sba.gov/about-sba/sba_performance/sba_data_store/web_service_api) has some nice APIs that are consistent and well presented:

Lasty, we have a couple of loose agencies to look at (or not):

Ok that was it. I know there are more APIs to look at, but this is derived from the master list of federal government developer portals. This gives me what I need. Although, I'm a little disappointed I have less than 5 Swagger definitions, after looking at about 190 APIs. 

After looking at all of this, I'm overwhelmed. How do you solve API discovery for a mess like this? Holy shit!

I really need my fucking head examined for taking this shit on. I really am my own worst enemy, but I love it. I am obsessed with contributing to a solution for API discovery that will work in private sector, as well as a public sector mess like this. So where the hell do I start?  More to come on that soon...


The edX API

This post should tell you about how behind I am in my storytelling—this story is from an event I attended in Arlington TX, on April 30th, and May 1st. While in Arlington, I spoke to a group of professionals who were crafting an online data & analytics course. A couple of the participants were from edX, the online course platform partnership between MIT, Harvard, UC Berkeley and other universities.

Over the course of two days, I had a cance ask the question, where was the edX API? Seemed like an obvious question, to which Emily Watson, the program manager at edX, responded, “Its on our roadmap”! An answer I get from many online companies, but Emily pulled up their roadmap on the wiki, and indeed it was on their roadmap.

I told Emily, I’d review the edX platform, and provide some feedback, that they could incorporate into their strategy. This one is easy, and is my basic feedback for any company with an online website and/or application—you make everything currently available on your site or application, available via an API. A quick glance at edX, these would be:

  • Courses - The 175+ courses available on edX, should also be available in a machine readable format, via the edX API.
  • Faculty & Staff - The 400+ faculty and staff involved in producing edX courses should be available in a machine readable format, via the edX API.
  • Schools & Partners- All edX partners should be available in a machine readable format, via the edX API.

You start with the low hanging fruit, by establishing three, simply designed web APIs for those already, publicly available resources, then move on to providing the essential business building blocks for any API:

  • Dedicated API Portal - Simple, dedicated portal for edX API integrations, that isn’t just for developers. It should be easy enough for anyone to learn about the edX API, without too much technical detail front and center.
  • Simple Landing Page - The home page of the edX API portal should be dead simple, explaining what the API does, providing easy one click access to get to whatever any API, or potential API consumer will need.
  • Self-Service Registration - The edX API needs to be accessible 24/7, and developers should be able to register without approval, and at least get minimum access to the system to begin playing with resources, even if it takes approval to get higher levels of access. Modern API management solutions like 3Scale work very well in managing this layer of access, and service composition.
  • Interactive Documentation - The standard for all APIs in 2014 is to provide interactive documentation, making learning about an API, a hands on experience. There are some common approaches to defining an API using machine readable formats like Swagger, which will automatically generate interactive documentation for consumers.
  • Code Samples & Libraries - Developers need all the help they can in getting up and running with an API, and right after interactive API documentation, code libraries and samples are critical to onboarding API consumers.
  • Blog W/ RSS - An active blog, with RSS for syndication provides the necessary stories and resulting SEO that helps communicate the value an API offers, and keeps new and existing consumers in tune with API operations.
  • Twitter Presence - An active Twitter presence is common place for leading API platforms, and edX will need an active Twitter account to support its other communications, and support operations.
  • Support Ticket System - An easy to use, trackable support system for the edX API platform will be essential to establishing a feedback loop with API consumers. Github issue management works very well, for an out of the box, API support ticket system for all public APis.
  • Discussion Forum - Discussion forums are common place in APIs, and provide potential indirect, community support, that can help new and existing API consumers find what they are looking for. Don’t limit yourself to a local discussion forum, there are SaaS solutions, as well as existing developer driven communities like Stack Overflow.

Next make sure and cover the basic political building blocks:

  • Terms of Service - Provide simple, liberal terms of service that incentivize integrations and development.
  • Content Licensing - Make sure all licensing for any content is explained front and center.
  • Code Licensing - Clearly license all client side code samples, libraries and event potentially server side code for the API.
  • API Licensing - Make sure the API definition is openly licensed, so that it can be re-used, re-mixed in many ways--consider putting into API Commons.
  • Branding Requirements - Provide clear direction when it comes to branding requirements around API integrations.
  • Rate Limits - Help developers understand what rate limits are in place to keep system stable, and what ways there are to get more access.
  • Pricing - Are there any costs for API access? If there are, make sure and clearly explain what is free, and at what points do resources cost.
  • Partner Tiers - Establish multiple layers of API access, allowing trusted partners to achieve higher rate limits, and write access to resources.

That should do it! ;-) As you can see the APIs themselves are just the starting point. It is too late for edX to be API-first, which would have been much easier, but you have just get your basic APIs, the supporting business and political building blocks, and get down to the hard work of evangelizing, managing, and iterating on the edX API. The learning doesn't start until you have the API up, being consumed, and have engage users who are providing feedback for the roadmap.

Once a basic edX API is up and running, and edX has some experience under their belt in managing the API community, then some thought can be put into creating a read / write API, allowing access to student information via oAuth, as well as letting trusted partners publish content, and student data into the system. A read / write system could be done in a single release, but since edX is a little behind curve on getting API up, I recommend cutting teeth on a read only system for version 1.0.

In reality, this advice for designing, deploying, managing, and evangelizing an API applies to any company with an online presence. It is a pretty proven formula extracted from watching multiple API leaders like Twitter, Google and Amazon. In this particular scenario, edX needs to get on the ball--it is difficult for me to imagine being an online education platform that operates between multiple universities in 2014, and not having an active API. ;-)


Deploying APIs Using Heroku And 3Scale Add-On

I am playing around with deploying APIs using multiple cloud platforms, and using popular container solutions. Next up is quickly deploying one of my utility APIs, to Heroku, complete with access control, traffic reports, and supporting analytics with 3Scale infrastructure.

Application - Simple Screen Capture API
I’ve been working through an operational harness for deploying all my APIs, and the API I use to push forward my approach to API deployment, is a screen capture API. This is what I will be deploying on Heroku, creating a simple application, that will take screen captures of web pages that I pass to it.

3Scale Heroku Add-On
To make API deployment on Heroku easier, 3Scale has created a Heroku add-on that you can easily deploy for any application you have running on the PaaS platform. With just a single click I added the 3Scale Heroku add-on to my screen capture API--now I can apply 3Scale API infrastructure to my API.

API Access Control
Using 3Scale I can control who has access to my screen capture API, by setting rate limits and quotas, and if I want I can even charge for access. I can compose any type of service levels for my API that I want. I will create tiers for internal access, external partners, and a public tier where I charge a fee for each API call.

API Traffic Report and Analytics
With all of my screen capture API traffic managed with 3Scale, I can now see traffic analytics and reports, broken down any account, application or for specific service. Any developer who signs up for my screen capture API, will get access to the reports for their account usage as well.

Ready-To-Go Developer Portal
One of the benefits of using 3Scale for deploying your API on Heroku, is that you get a developer portal to go with your APIs. This is where developers will to learn about your API, register for API keys, and view reports on their usage.

Free To Get Started
One of the main reasons I use 3Scale to mange my APIs, is that even though I’m on the paid tier now, 3Scale allows me to fire up new APIs, manage access and receive reports at no charge. I don’t pay until my APIs start generating traffic, and if my business model is sound, I should be covered. You get up to 250 user accounts with a traffic volume 150,000 API calls per day before you have to move up to the next tier.

Multi-lingual APIs In Cloud
3Scale provides instructions for deploying APIs using Ruby, Node.js, and Java, allowing you to the language you prefer for API deployment.

Drive API Code From Github
I will be driving the development of the code for my screen capture API using Github. Heroku allows me connect my application to a specific Github repository, allowing me to manage all my API deployment centrally using Github.

I already have a PHP version of my screen capture API, that uses the Slim REST framework, which wraps around PhantomJS, which does the actual screen capture. Now I can take my standard utility grade, screen capture API and easily deploy on Heroku, for my own internal use, for a partner, or to any wholesale customer who would like to deploy a screen capture API in their own Heroku account.

Next I’m going to look at various ways to deploy the same screen capture API using Amazon Web Services (AWS). There are several options for deploying APIs on AWS, and hopefully I can find the easiest, lowest cost way to get an API deployed, by playing with several approaches.

Disclosure: 3Scale is an API Evangelist partner.


If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.