Turns out mine is bad. I’m also uncool and old. More on that in a bit. Let me tell you how I found out…
I’m a big fan of Pudding.cool. I love their visual storytelling. It’s one of the handful of sites I’ll type in by URL to visit (yep, like an old guy). If nothing else, you should sign up for their Winning the Internet newsletter, which is where I found the link to their How Bad Is Your Spotify? project. I was just thinking how bad Spotify had been lately at suggesting interesting playlists, so I eagerly clicked over, hoping to find out why the music service had lost a step in its personalized recommendations.
Instead, I found there was a different concept at play:
Our sophisticated AI judges your awful taste in music.
I recognized this idea immediately. Quickly judging someone’s taste in music was something I often did in my head at the record store where I worked during college, The Disc Exchange (RIP). Like The Pudding’s Spotify project (and every 20-something hipster), I also started from the position that most people have awful taste. To be clear, the Disc Exchange wasn’t Other Music and there wasn’t a ton of attitude at the store. It had more to do with my youthful bias and an unsettling exposure to the music-buying public that conditioned me to assume the worst in people after awhile.
Here’s how it would work: I would make my one- to two-minute assessment of that a person’s music tastes based on what they asked me, what section of the store they were browsing, or the CDs they held in hand. If I concluded this person was “basic” on this limited amount of situational information, then they would get an “Oh, I bet you’ll like the new Pearl Jam” response when they asked, “Got any recommendations?” That would have been the Vs. album btw, a more than fine recommendation. The thing was it was lazy and loaded with smug attitude. I regret college record store me. But I also realize that was part of the culture at the time.
So when the fine folks at The Pudding figured out how to give chatbots record-store smugness, I thought “Finally, a AI use case I can understand!”. The folks at The Pudding are smart, so I assume this whole music taste project was done with their tongues firmly placed in their cheeks. What is more interesting is that on a meta level, they have just demonstrated how biased “artificial intelligence” is due to the people who make it and the data that goes into it. Let me explain.
How good is your training data?
Like all good machine learning projects, The Pudding developers were using OBJECTIVE DATA to train their machine learning model to judge your Spotify listening history. If you take Pitchfork, Brooklyn Vegan, and hundreds of other music blogs and amalgamate all that coolness into a bona fide artificial intelligence algorithm, well, all you got are FACTS….right? I even clicked the fine print to ensure there was some legit tech testing my listening worthiness:
Oh, good, they also included subreddits! Clearly this was comprehensive science, so I was in. I logged in to give them access to my Spotify account. To ensure this process was free of misinterpretation or bias, I was prompted with qualifying questions:
I thought it was a nice touch that they let the user know the AI was architected by real music snobs with the “lol” and “omg” comments upon looking at my collection—er, Spotify listening history. I lived in fear of my wall of CDs being judged by people that came to my apartment. That’s because I also judged people by their collections. “So, you own every album by Marillion? Fascinating.” (Note: That would turn out to be cool AF after all. Another hipster dilemma for another post.)
I did get a little irritated by the ironic question because I’m a Gen X’er whose Alanis and Reality Bites exposure instilled fear of misuse of that rhetorical device at a formative age. What I do know is “Down the Dream” is a badass song by these badass sisters. Check it out:
So I double-checked. Nope, I wasn’t listening to Maggie and Terre Roche ironically! In fact, I went down to Mill Valley Music and bought a vinyl copy of that record. Gary, who runs the store, is literally the nicest guy in the world and would never judge you or your tastes—you know, like it should be.
As the AI plowed through my listening history, it discovered other indicators of my Spotify awfulness:
Wait… the Grateful Dead are cool now, right?! Phil Cook is definitely cool. Is being a Lambchop stan a bad thing? Last time I saw them in before times, they were still playing small theaters full of hipsters. I wasn’t sure, but I still had a bit of confidence the AI was going to figure out that this ol’ record store clerk still had it.
Wait…is listening to Mount Kimbie a sign I’m depressed or something?! It’s not Swedish Death Metal, which I would think of as truly morbid. But now I’m at the age when I have no idea about anything anymore. Maybe all of music is a bit miserable like Hornby wondered:
“What came first—the music or the misery? Did I listen to the music because I was miserable? Or was I miserable because I listened to the music? Do all those records turn you into a melancholy person?”
—Nick Hornby, High Fidelity
But now things were getting really awkward. I didn’t want to fuck, marry, or kill Hiss Golden Messenger, Lambchop, or Bob Dylan. Although I should point out that all of them seem reasonably suited for your sexual or life partner needs. I want all of them to live a long time, too. This felt like we were training the AI to fear us…again. But I was so invested in this test that I couldn’t back out now. I put my hands over my eyes and start clicking until the test moved on.
I got an initial “Hey, your Spotify looks great!” message. Yay! But then…the AI changed its mind, erased that sentiment, and started to tell me what it (and its creators) really thought:
While I will freely admit a Stones addiction, there was only one week where I listened to Nothing’s Shocking two or three times while running (BTW, still great!). I had to double check what they meant by stomp and holler to know why that was a knockdown. Actually, that categorization makes sense as I’m all over that playlist…but none of those artists are from the mid-’90s. Or mainstream country. Or necessarily yoga-inspired for a nag champa burndown. Weird.
What is signal and what is bias?
What this exposes is a bias toward whatever data The Pudding’s AI got from Spotify and the biases they put in to train the data. What is the right number of times to listen to something that signals that a piece of music defines me? I think this is part of the Spotify recommendations problem too. How does Spotify balance my “liked songs” with songs I play repeatedly? Then how do they make recommendations from my listening cohorts and look-a-like users when I only wanted to listen Jane’s Addiction for a week of nostalgia? What about that day I wanted to figure out what Hyperpop was…but didn’t like it? It can get complicated quick.
As my Spokestack data scientist friend Will Rice said, “The important take home is that algorithms are sold as objective but are actually subjective because of the selection bias for recommendation features”. To Will’s point, I’d love to look through the training data to figure out where Jane’s Addiction or The Roches set off warning bells of my music stankiness. Is Gorilla vs. Bear just hating like crazy on The Roches? Somewhere, someone made a decision about what was cool based on year, genre, etc that echoed through the whole AI’s “objective analysis”.
And with chatbots, the developer biases are always clear. You don’t think the AI learned these ageist and stereotypical putdowns on its own, do you? I mean, I don’t read all the music blogs they’re ingesting to train their machine learning model, but are these the things you say to someone whose music doesn’t jibe with the new arbiters of cool?
Nope. Engineers put those phrases in little reference YAML files in a conversation manager they whipped up for the chatbot to use as responses based on the inputs from my “bad Spotify.” No magic here, folks. Just good ol’ social engineering.
So let’s talk more about the inputs that categorized me as such a stereotypical loser, and remember, this was derived from the objective data!
Well, it’s hard to dispute the data. I love all those artists and songs, so color me awful. What’s interesting is that these results demonstrate where the data inputs fall apart. Here are the telltales:
- The default playlist problem: “Hawk” by Brasstronaught was the first song on a playlist I made in August to listen to while running and working out. Out of sheer laziness and a pure self-hatred if I pick a “power workout” playlist, I kept playing my workout playlist over and over again this fall. In fact, four of the five songs on my “too much” results come from this playlist.
- This song is the first track on new album problem: The only track in my top 5 that isn’t on that workout playlist is the Mount Kimbie song, “Four Years and One Day”. That track is the first track on their album Love What Survives. Two years after its release, I still listen to that album as a default when I can’t think of anything else to listen to because it is brilliant. Unfortunately, that track is not even close to my top five favorite songs on that album, much less in general.
- Another default album problem: I really didn’t listen to a lot of Lambchop this year like I have in years past, and I did it purposely. Their album Is a Woman was my go-to for focus or sleep for a long time. Why did I stop actively listening to it? For the last few years, Lambchop’s “Is a Woman” came up as a top album for me in my “end of year” Spotify reports. It angered me that I was paying Spotify to rent an album that I own on vinyl, CD, and FLAC formats. More on that issue in another post.
- Autoplay and the artist who has 500+ albums problem: The thing about Bill Evans, Bob Dylan, and the Grateful Dead is they have a lot of albums. So listening to them or, more often, letting them keep playing while I do other things overweights their influence on my listening output. If I put on Bill Evan’s Undercurrent and start doing other things, 3 of his albums can play before I turn it off or listen to something else.
My point is the data is more influenced by how I listen to music than by why I listen to it. Continuous play mode, mobile access (“going for a run”), and sheer passive listening brought on by cloud-based music have muddied the signal of what we like and enjoy listening to the most. If you got into my car when I was in high school, it was clear from the wear on my cassettes that The Replacements’ Tim, Husker Dü’s Candy Apple Grey, R.E.M.’s Life’s Rich Pageant, and The Cult’s Electric defined me. I voted with my limited amount of money and space. Now, with unlimited storage for $9.99 a month, there are no clear signals of what I really value unless you come over and see my vinyl collection. Again, like an old.
Figuring out what data we are sending that truly contains the signal of our likes and dislikes will always be challenging. Netflix, Amazon, Spotify, et al struggle with it as their data catalogs get bigger and our consumption becomes more ambient than active. All of these companies choke over and over on recommendations due to the overabundance of non-contextual inputs and data complexity. It’s hard to make personal recommendations at scale.
This is why I get most of my music recommendations from Worldwidefm.net, Aquarium Drunkard, Bandcamp Weekly, and Mixcloud DJs. Humans still make better playlists. It is also why I think we’ll see a comeback in favor of human-based recommendations. Even Spotify is betting on humans to make better playlists and why they launched a service that enables everyone to create podcasts with licensed music late last year.
What The Pudding really judged was my “newness,” which was pretty weak this year. This was more clearly pointed out in my Last.fm recap:
So while I’ll own my flannel-shirt wearin’, mid-’90s craft beer persona proudly, I do get the point that I need to up my new-music game. Anyone got any recommendations? No AI’s allowed.
Hat tip to Ben, Monteiro, Om, and Drew, who all inspired me with their physical ‘zines (YES), blog posts, and newsletters. All of you contributed to my development and overall happiness throughout last year with your insightful, heartfelt writing. Thanks!
Really enjoyed Fred Wilson‘s post “Grinding“. Everyone wants to talk about “hustle” when building a company. In reality it is usually more of a grind day in, day out until you figure it out – or die. I especially appreciated how he addressed the “magic bullet” theory to fixing company issues in this line:
But it is rarely one thing that a business needs to succeed. It is often a little bit of everything.
Lot of truth packed in that quote. Fred references his experience with Twitter’s early struggles with their fail whale and explosive growth and popularity. That is definitely one kind of grinding…managing a rocket ship as tries to reach escape velocity.
What I wouldn’t give for that kind of grinding. There’s another kind of startup grind, and it’s one our company has been doing for the past 3 years. We’ve been searching for product/market fit in Voice User Interfaces (VUI) since inception. When we started, we targeted smart speakers with branded services to catch the explosive growth and popularity of smart speakers like Alexa. Having watched how web and mobile changed how consumers interfaced with content and services, it felt inevitable that voice was the next big interface change and smart speakers were the next big wave.
But sometime it takes a while for waves of change to reach the shore. Here we are with millions of smart speakers units shipped and five years after the launch of Alexa, and we’ve had no break out VUI brands or companies. The biggest success story in smart speakers is a guy who works at Ford who bought a Tesla with cash with his Alexa skill he built on the side. That’s a nice side hustle that helps one developer and Alexa, not sure it’s a business that continue to grow.
So we stopped focusing on smart speakers and making a transition to mobile voice. We’ve been grinding over the last year to help mobile apps develop their own VUI, including a whole company rebranding. Making this change has definitely been more of a grind than a pivot. That’s because we are refining the value we create for VUI than completely giving up on it.
We’re excited about enabling the mobile voice movement. There are implications for better privacy, brand building and revenue streams that is available in smart speakers alone. Hopefully, our grinding will pay off and our efforts to “change a little bit of everything” about our company will help us find success.
Please keep your fingers crossed for us! And if you app needs a voice, please shoot me an email.
You better think (think)“Think” by and co-written by Aretha Frankin
Think about what you’re trying to do to me
Think (think, think)
Let your mind go, let yourself be free
So I was thinking about a bunch of stuff this weekend. I spent a lot of time thinking about Aretha. I even made my kids watch The Blues Brothers so they could see that she was more than an amazing singer, songwriter and activist. She was funny. For me, she stole the movie with this scene and I’ve thought that since I first saw it as a kid…about my kids age.
Let’s go back, let’s go back
Let’s go way on way back when
I didn’t even know you
You couldn’t have been too much more than ten (just a child)
I ain’t no psychiatrist, I ain’t no doctor with degrees
But it don’t take too much high IQ to see what you’re doing to me
I’ve also been thinking about social networks (along with a lot of other people) and how they are having a negative affect on me and those around me. The negativity is mostly coming from a small group of people and millions of bots who are spewing a bunch of vitrol in their posts and videos and it’s hard to figure out what to do about these bad actors on these platforms.
The debate concerns who and who shouldn’t get banned for saying hateful things or what should be considered offensive. I worry about asking the folks who run these networks to make decisions on these issues that impact our society and culture is a bad idea. Don’t get me wrong, these are really smart people. I’m just a bit weary of business people setting cultural and societal contracts instead of folks who dedicate their lives to public policy and service. I’m sure people are on the payroll at these companies that have dedicated their lives to such issues, but I’ve rarely seen internal policing go well. Maybe it will.
Ten or so years ago when I started on Facebook and Twitter the services were shortcuts to get information on my friends (on FB) and on tech, music and sometimes sports news from Twitter. It seemed like they were mostly link services that like my RSS reader, made it a bit easier to track a lot of things. Hyperlinks are powerful and I’d bounce around from source to source. The signal was distributed and un-algorithmed. Somewhere along the line that changed…at least for me. Pre-Twitter, I got my the majority of my information from visiting blogs and large media companies directly instead of quick-take tweets. Then I noticed I let the quick take replace reading the original piece altogether. I purposely had to make an effort to subscribe and visit a few publications and blogs every day after the election to change my news consumption habits.
With a lot of people frustrated with Twitter’s take on censorship, some folks were floating quitting Twitter or trying other services like Mastodon as an alternative. I tried Mastodon a year or so ago, but it was a bit too much of a headache to use and it didn’t seem like that many people were using it. On Friday, I tried it again and found it much easier to use. And there were a lot of people using it, at least on the instance I joined. Does it feel like it could replace Twitter tomorrow? Nope. But Twitter didn’t seem like it would replace the news for me either when I first joined.
I’m not going to quit Twitter as there is still content value being created for me. I doubt that the network value of Twitter will go away for several years. But maybe Mastodon will pull out my “music Twitter” so there’s a place I can go that is higher signal on music and not crowded out by tech, politics or the people who conflate Twitter posts with Facebook posts of their personal lives non-stop. Twitter may not being growing significantly, but the noise has increased for sure.
I do think Mastodon or some distributed variant has a legitimate chance now to pull attention away from Twitter. Or at least some of the more interesting contributors, especially when I see two of the first people I followed on Twitter, Andy and Anil, are there and posting regularly. You can also kind of see the potential for some new community/publication thing forming when you see people like Will , Gary and Jeff playing around with Content.Town – their own Mastodon instance. Sure it’s a joke. Twitter seemed like one at first too.
So back to what Simon tweeted yesterday. Why don’t we just start blogging and rolling again? Why not go back to making content on my own, er leased, servers? It was never that hard to make a blog and it’s even easier now. Within a few minutes, good ‘ol Dreamhost (proud customer since 2001) and WordPress had me back up again. So I’m going to start cranking out blog posts again even if only two people read them. I’ll tell you, it’s much easier to write and read a blog post than a tweet storm.
And to Simon’s point about subscribing to each other’s Atom/RSS feeds. We can still do that with services like Feedly, even if Google Reader is gone. Or maybe we should call Mark Fletcher to stand up my original fav Bloglines again. Or maybe there’s something new that sits on top of Twitter, Mastodons and blogs that is distributed and free of algorithmic feeds that more evenly distribute our attention and dollars to creators. Anyway, you can find my RSS feed in the menu or I’ll keep plowing links on Twitter.
Finally, thanks for reading my post. I so wanted to title it Back to the Future, but I had too much Aretha on my mind not to bring her into it. If you want to read a great piece on Aretha, my favorite is from Patterson Hood on The Bitter Southerner. Which is a great independent blog you should be reading.
Three predictions on virtual assistants in a week full of predictions — from folks like us who actually build them.
As voice assistants dominated CES last week, people are asking “is every toaster, microwave and car really going to talk to me now?” Or are virtual assistants just a fad since many are being shutdown or proclaimed dead?
Prediction #1: Google and Amazon will be in a lot of devices, but neither will “own” the voice entry point to customers.
That’s because the data and brand interactions created by conversations with customers will be so valuable that companies will not want to share this data with potential future competitors. Companies might experiment with Alexa and Google, but they are not going to totally give up on building their own voice platforms as Ina Fried at Axios observed this week at CES:
It’s a fierce battle between Amazon and Google to get their assistants included on other companies’ devices. At the same time, hardware makers including Samsung, LG and Roku are (also) putting their own voice assistants into their products.
As these companies watch their users interact conversationally with their hardware, they may find that the verboseness required to be a “smart speaker” like Alexa that can answer everything may not be required to enable useful conversational navigation of, say, a microwave. A smart speaker needs to be an omniscient service that can answer everything for everyone. Often a few basic user intents like ”Yes”, ”No” and some base navigational phrases can help the user accomplish most tasks — especially if the device has a screen that can convey information as well.
I’m completely biased on this idea, but I believe companies will start turning to dedicated (plug!) conversational developers to build holistic voice and chatbot features for their businesses outside of just embedding inside the big tech NLU platforms like Alexa or Google. To be clear, I believe these proprietary services will have interoperability with the major NLUs, but will be something more than “just an Alexa skill” in the near future.
Also, the companies that are experimenting with their own services now will be way ahead of the curve when their customers expect having a personalized conversation with a brand as a primary feature. Experimentation while the market is still growing and the bar to wow the end user is low is important.
That’s because even though Google, Amazon and Apple have a huge ASR lead on most speech recognition services, the most practical assistants do not need to understand a massive vocabulary to accomplish most tasks. Again, that’s because a single domain application can manage to handle the recognition of limited entity names (cocktails, movie names, etc) within a reasonable amount of time. It can be a bit rough at first, but better to work out these issues now while the number of total users is small. If you were late to the web or mobile, don’t blow it this time by waiting to find out how to present your brand in the conversational internet. Do it now while the stakes are low.
So expect even more voice and chat platforms outside of Amazon, Google and Facebook to exist and thrive in the marketplace over the next few years.
Prediction #2: The conversational internet will expand and interoperability between conversational platforms will accelerate as consumers demand consistent, state-aware conversational relationships with their favorite brands across platforms.
Even if I’m wrong and there are not dozens of successful conversational platforms and only 3–5 conversational platforms dominate, the consumer will demand that the relationship with their favorite brands transfer state to whatever platform is convenient for them. Think streaming a movie on Netflix on multiple devices for conversations.
For example, I may start talking to Alexa about a recipe in my kitchen in the morning, but I may want to pick back up the conversation on a Slackbot while at work to confirm I want to make that recipe. Then I might want to pull it up on my phone when I’m shopping for ingredients later. If I have to start the conversation over again or the AI doesn’t remember what we last spoke about — even if that’s on another device or platform — that user will be irritated with the brand and think it is dumb.
There’s no reason for the consumer to ever feel like a brand is dumb just because it can’t remember what the last interaction was on a different platform. Platform lock-in will not work on the conversational internet, much like it didn’t on the web or mobile. Brands and their consumers will force openness onto these conversational platforms. The platforms that try to keep brands locked in to their platform will ultimately fade while the open ones expand…as usually happens. Can you imagine how pervasive Siri would be if it was launched as an open platform? MBA’s will be writing cases about that missed opportunity for years.
The thing that is really missing for conversational services to explode is an open standards body that will enable developers and companies to build interoperability for the conversational services. More on that in another post.
Prediction #3: Conversational assistants and chatbots are not overhyped or dying.
Sure, conversational assistants and chatbots have been kludgy (or downright offensive) in this first wave, but that doesn’t mean they are “dead” or dying. It means they are evolving. Yep, this is evolution and we’re seeing the extinction of things that aren’t quite right or fully baked.
Every major platform change starts with weird experiments or just plain bad ideas. 98% of startup products deserve to be evolutionary fodder. Another 1.5% were genius ideas that were too early. The other .5% that survive become monster businesses.
In Wired’s reporting on the “death” of Facebook’s M assistant, they lopped in “and so are chatbots” in the headline. In the vicious hype cycle of new technology, it wouldn’t be a cycle if chatbots and voice apps didn’t suffer a bit of a blowback after last few years of exuberance for those technologies. But to say that virtual assistants and chatbots are dead because the first wave of these applications are a bit wonky would be short-sighted.
M's core problem: Facebook put no bounds on what M could be asked to do. Alexa has proven adept at handling a narrower range of questions, many tied to facts, or Amazon's core strength in shopping. Another challenge: When M could complete tasks, users asked for progressively harder tasks. A fully automated M would have to do things far beyond the capabilities of existing machine learning technology. Today's best algorithms are a long way from being able to really understand all the nuances of natural language.
These two paragraphs completely wrap up both the promise and the problem with assistants. As users, we so want them to work! The user immediately goes to superuser mode with conversational assistants -whether chat or voice-based and ask them to solve all kinds of problems out of the scope of what’s currently possible. Inevitably, the user then curses the assistant when it fails and claims “this is stupid”.
So while the Pangea “all things for everyone” assistant phase ends, I believe we will move to a “continental drift” phase of assistants where smaller assistants will breakout and tackle complex domain-specific problems successfully for end users. There are already quite a few productivity and work-related chatbots that are effectively solving problems for customers. As more companies have their assistants focus on domain-specific or single purpose assistants we will see more consumers asking “why can’t I do that for (X problem)?” in their lives. Once this happens, I really believe every website, app, device and brand had better be conversational — or start the slow fade to oblivion.
A lot of these predictions came from the work we are doing at Pylon ai. Our first two beta products, Tasted and The Bartender, are popular voice apps on both Alexa and Google. Our apps are built to work across voice and text platforms, so they also work on FB Messenger and Slack. Our apps are also “multimodal”, which means you can use them with a screen when it’s easier than talking to them. You can see a video of how that works here. If you would like your own cross-platform, multimodal assistant for your business, please email me.
And… if you made it this far, we owe you some schwag or a Google Mini! If you’re interested, send us your address! Or if stuff is not your thing, please sign up for our newsletter here for updates on conversational assistants, Elixir, React and other stuff we talk about at Pylon. Thanks!