Quantcast
Channel: Apple – import digest
Viewing all 69 articles
Browse latest View live

Week 7

$
0
0

Pick of the Week – what does a CTO do?

Chief Technology Officer is a coveted executive role that many senior techies aspire to attaining.  The CTO is broadly the ultimate decision making authority for all things tech within a business and in some senses its most capable technologist.  However, there is no single unifying job description and CTOs come in all sorts of shapes and sizes.   As this post points out, it’s not really about being the smartest or most experienced software engineer, it’s ultimately about being the best technology strategist for the current stage of evolution of the business which helps explain the varied background of incumbents:

“My advice for aspiring CTOs is to remember that it’s a business strategy job, first and foremost. It’s also a management job. If you don’t care about the business your company is running, if you’re not willing to take ultimate responsibility for having a large team of people effectively attacking that business, then CTO is not the job for you.”

Many of the territory a CTO operates in would benefit from the application of hindsight such as this acerbic Quora response outlining the what the author wishes he’d known at the start of his career:

    • The most important job in your company is sales.

    • What the customer wants isn’t important, it’s what the customer is willing to pay for that is.

    • Management is the only thing standing between you and unemployment.

    • Most MBAs aren’t “Masters” of anything.

    • There is no substitute for downtime.

The dismissive comment on MBAs reflects a tension often observed between business and technology staff.  MBAs tend to be given more credence particularly within large organisations but the qualification alone is often not as useful as having deep technical understanding of the business you’re in.  Or, as the author puts it:

Don’t ever get bamboozled by an MBA, believing them just because they have an MBA. If it doesn’t make sense to you, it probably doesn’t actually make sense.”

Devices and Manufacturers

  • The extent of the struggle for profitability amongst Android OEMs across the board is outlined by Charles Arthur here.   He provides some interesting data on how little profit they are making relative to Apple and the downstream problem that creates in terms of cash to invest in meaningful differentiation:

“there’s little opportunity even for high-end Android OEMs to invest and innovate, because it’s not profitable enough”

  • There seems to be little in practical terms that Google can do intervene to alter this dynamic:

  • Arthur also points out ABI Research data indicating that Android shipments appear to have had their first ever QoQ decline. If so, it would represent a genuine inflection point and potentially accelerate efforts by OEMs to consider alternative OS platform propositions.  Two such candidates would be Ubuntu Phone and Tizen.

Android shipments fell in Q4 2014, ABI says

“Scopes are essentially contextual home-screen dashboards that will be much simpler and less time-consuming to develop than full-on native apps. They function by presenting related content from different providers alongside each other on one screen. .. The user experience offered by Scopes seems a lot more intuitive than the “app grids” that dominate most devices. Apps offer distinct, siloed chunks of information or content. But what users are really after is the content those apps bring them. If apps are content silos, Scopes are windows into each silo showing exactly what you need when you need it.”

  • The Samsung Z1, their first Tizen phone, is intended to be their best candidate to break the Apple-Google duopoly on mobile. This hands-on review suggests it still feels like “a bad Android clone“.
  • The strategic goal for Tizen is to get to the same place Google are already at with Android, namely locked down but expressive native and web APIs for application developers with open source underneath for ease of porting:

  • However it’s been a tortuous journey so far over the last decade with many tributaries leading to Tizen. The Samsung Z1 runs Tizen 2.3 with a 64-bit version 3.0 being worked on which Samsung hope will become the “Highlander of the mobile world“. However the family tree below doesn’t take into account another industry mobile Linux initiative namely Firefox OS which added Qualcomm SoC and LTE support in 2014:

“The Linux-based OS is an amalgamation of every other failed or aborted Linux smartphone platform. If it’s Linux-based and not made by Google, there’s a good chance it’s been rolled into Tizen at some point.

“We want to move from people needing Windows, to choosing Windows, to loving Windows,” said Nadella at a recent press event for Windows 10. “That is our bold goal”.”

Apple

  • Fascinating insight into how Apple’s power can make or break even a large component supplier, in this case touchscreen manufacturer Wintek.

“the Mac has been outgrowing the Windows platform for 34 out of the last 35 quarters.”

Screen Shot 2015-02-08 at 5.35.43 PM

Google and Android

“In 2005, nobody really knew what would come of online maps, or how they would become such a crucial aspect of daily lives in the Internet-connected world.”

  • YouTube is also 10 years old this week and according to The Guardian has morphed into “the most interesting place on the internet“.  Here’s the first ever clip:

Apps and Services

  • According to DeveloperEconomics “90% of developers have implemented a third party analytics SDK into their app” and yet “only 5% of these developers knew what to do with the data points which they were tracking“.  They’ve published a guide reviewing some of the key metrics developers should take note of from Cost Per Install (CPI) to Session Length, Time Spent in App, DAU and MAU (daily|monthly active users) to Lifetime Value (LTV) and Average Revenue Per User (ARPU),
  • Flipboard have finally introduced a web app property having achieved all their success to date through their mobile only suite. Some are contesting whether it is a real mobile web app because it is built around content rendered entirely in Canvas.

Asia

tech in asia data

  • Alibaba announced a $590million stake in domestic OEM Meizu which has around 2% market share in China.  It’s an attempt “to expand its mobile operating system in a shrinking, cut-throat handset market.
  • By way of example, Chinavision are offering a Siswoo 4G LTE Mediatek MTK6595 octocore Android 4.4-based smartphone called the Monster for sub-£200:

  • It maybe a cutthroat market but as TechInAsia highlight, there is an upper ceiling of around $480 that Chinese domestic OEMs cannot seem to break which is precisely the high end luxury niche that Apple are prospering in:

“the rise of Chinese brands in the smartphone market does have its limit. There is one place that Chinese brands, at least for the moment, simply can’t go. One barrier that no one has been able to successfully break. That barrier? RMB 3,000.”

  • WSJ have picked up with the news recently reported that Apple is China’s top luxury brand with seemingly more headroom for growth in terms of brand, geography (specifically Western China) and new product possibilities:

“Apple’s positioning in Chinese consumers’ minds is getting stronger. Apple Watch should definitely benefit from this position,” said Neil Shah, analyst at Hong Kong-based Counterpoint research.

  • Counterpoint also reported that the three biggest markets for smartphone growth in Asia outside of China are India, Indonesia and Bangladesh.

Networking

“If we consider that the ant colony’s goal is to collect more food and expend fewer ants, and a server’s goal is to send a file and avoid congestion or overload, then the similarities are clear. Sending a packet through the internet is analogous to releasing a forager ant into the wild. Getting an ack of a packet’s receipt is analogous to a forager ant returning with food.”

  • The post goes onto to outline a range of biological analogues for human algorithms:

Machine Learning and Big Data

  • In another application of facial recognition technology that raises uncomfortable questions, crockpotveggies.com posted this admittedly impressive technical hack.  It involves automating Tinder to identify the k Nearest Neighbour ‘eigenfaces’ relative to a facial model built from likes/dislikes followed by the use of NLP to initiate and control follow-up messaging interactions.  The most disturbing aspect is how successful the bot has been at harvesting potential dates.

View image on Twitter

  • The deep learning applications highlighted above involve predictions made with unsupervised or reinforcement (aka semi-supervised) machine learning techniques such as clustering or genetic algorithms.  Predictive machine learning, however, is a broader discipline which also takes in less glamourous supervised approaches such as regression or Bayesian classification.

Machine Learning 1

Smart Retail

“Plaso appears to be using Bluetooth beacon technology to detect Google’s digital payments apps on smartphones physically in the store or near check out.”

  • Excellent and insightful post from a member of staff at the Brooklyn Museum responsible for a large scale deployment of iBeacons there.  It covers a range problems encountered from batteries running down to beacons detaching from adhesive.  The museum ended up using this “beacon installer cart” to aid the process:

My hacked out beacon install cart complete with cookie tins.

Wearables and the Internet of Things

  • This Benedict Evans post on the enduring attraction of the tangible in an increasingly digital world represents a timely starting point for the section this week.   For Evans, “connected objects” have a place particularly in a world replete with cheap smartphone supply chain components.   Mobile apps too have a physical aspect for him:

“There’s a sense that by getting an app you have somehow captured what’s inside it – pinned it to your home screen like a collector’s butterfly. … An app is tangible – it’s a thing that you have and keep and can’t lose or forget. “

Ruby

  • Another US startup, Trellie, are offering a compact white label smart Bluetooth LE module that can be easily mix and matched into a wide range of personal accessories.   It should prove attractive to fashion brands looking to get into the smart jewellry space:

“Trellie makes a small “nugget” around 15 mm x 10 mm in size that can be moved in between different accessories, like rings and bracelets. In conjunction with a mobile application, Trellie’s smart jewelry can light up or vibrate when you’re receiving calls or texts, or it can notify you of calendar events, and soon, more.”

Apple Watch to come in 34 designs

Raspberry Pi 2

“It turns out that if you try and take a photo of the Raspberry Pi 2 using a certain type of camera, it crashes and turns off. “

“the effect only happens under VERY specific circumstances. Flashes of high-intensity, long-wave light – so laser pointers or xenon flashes in cameras – cause the device that is responsible for regulating the processor core power to get confused and make the core voltage drop.”

Cars

“The company has hired at least 150 former Apple employees, more than from any other company, even carmakers.”

“Car research could be another step in the same direction [as Apple Watch], with Apple exploring how its brand might translate to premium goods beyond just the home computing market.”

developing a smart body panel that uses a grid of foil-like pressure-sensitive electronic sensors (plus special algorithms) to detect when someone dents or scratches your vehicle.”

Work and Culture

  • This Fred Wilson post introduces the Carlota Perez framework for understanding technological revolution in the context of capital markets.   In that framework, crash and recovery always takes place between early adoption and mainstream acceptance:

perez-cycles-final

  • Will the 21st century be defined by techno-data religions or recidivist religious zealots?  Or are they ultimately the same thing?  An interesting Google tech talk raises a number of profound considerations.  A key one being that many of the most difficult questions society faces in the future regarding topics such as genetics, algorithms, robotics and nuclear energy are entirely outside the scope of mainstream religious scripture:

  • Visualisation of fear of death vs. actual causes of death:

  • There was another illustrious ten-year anniversary this week to accompany those of Google Maps and YouTube.  Nathan Barley aged ten felt just as relevant and if anything an under-exaggeration in terms of what has come to pass since 2005:

  • Another fantasy that hasn’t dated quite so well is the belief that globalisation is a rising tide that uniformly lifts all boats.  The reality as pointed out by Pew Research is that real income growth has not been evenly distributed.  In fact for the very poorest and developed world middle-class, income growth has been static or in decline over the last 20 years or so:

Source: Milanovic, B., Lead Economist, World Bank Research Department, Global income inequality by the numbers.

 


Week 8

$
0
0

Pick of the Week – Exponential Organisations

Large organisations are increasingly being impacted by a changing market dynamic privileging smaller, more agile alternatives. These nimbler competitors are typically unconstrained by legacy concerns and are able to fully exploit new technology and a wide array of services that have sprung up within the App Economy over the last few years.  By doing so they are essentially able to disrupt the one major advantage the corporation has enjoyed since its arrival, namely the ability to preferentially minimize transaction costs within a market.  This is the territory explored in a Medium article which suggests that many large firms are now struggling to compete with the collapsing transaction costs that Internet-enabled startups are able to extract:

“The Internet is nothing less than an extinction-level event for the traditional firm. The Internet, together with technological intelligence, makes it possible to create totally new forms of economic entities, such as the “Uber for everything” -type of organizations that we see emerging today. Very small firms can do things that in the past required very large organizations.  … We stand on the threshold of an economy where the familiar economic entities are becoming increasingly irrelevant.”

The argument echoes one of the key points made in a recent Singularity University book “Exponential Organisations” which ought to be essential reading for anyone working in the modern tech landscape.  The book examines the structure and operational practices of some of the “unicorns” that have exploded onto the world stage in the last decade – the likes of Uber, Snapchat, Xiaomi and AirBnb. These companies have consciously adopted a culture built around small trusted teams, transparent communication, open innovation and judicious use of virtualised infrastructure.  They’ve been able to leverage that base to build global scale propositions with millions of users within a very short period of time.  According to the author, such “exponential organisations” are the harbingers for a new mode of working albeit one that requires dramatically fewer employees:

Apple, Watches and Cars

  • The New Yorker ran an extended piece on Jony Ive covering similar territory to that explored by Leander Kahney in his fulsome biography of Apple’s design chief which labelled him a “genius”.  There are some interesting new perspectives provided by the New Yorker, not least Ive’s predilection for luxury products and top spec cars.    Among the revelations, it seems that Ive’s taste for luxury extends to an interest in luxury smartphones, not just watches:

jonyivevertu

“The more I think about it, and the more I learn about the watch industry, the world of luxury goods, and the booming upper class of China, the better I feel about that bet. I don’t think I was wrong to place a friendly late night bar bet on a $9,999 starting price. I think I was wrong to guess just $4,999 in my ostensibly sober published analysis.”

FRANCE-CHRISTMAS-FEATURES

  • In relation to the cars, although they increasingly resemble “computers on wheels”, Apple will have their work cut out to disrupt an industry that is already being disrupted by urbanisation and smartphone usage to the extent that “car sales may never recover to their pre-recession peak“.  This fascinating graphic of driving licence penetration by age over the last thirty years shows a marked decline among younger age groups and a corresponding boom in the over 50’s:

  • BBC Radio One DJ Zane Lowe is joining Apple.   They’re clearly planning something ambitious with recent acquisition Beats Audio potentially around music streaming.  Bringing Lowe on board suggests that curation via taste-makers will be a part of the picture.  Jony Ive apparently likes “loud techno” rather than the mainstream rock that Lowe favours.   Expect to see a 90’s Techno Masterpieces offering as part of the mix.

Devices and Manufacturers

“most hardware makers aren’t good at software. By partnering with a truly open operating system — as opposed to a locked-down ecosystem — device makers begin to capture more value, and share revenue in places they haven’t been able to play before: app stores, mobile services, advertising, and other ways to monetize on phones.”

  • Xiaomi’s MiUI is another contender and is apparently “being used on 347 different handset models from different brands“.  Wired suggest the MiUI’s service integration and feature iteration will influence the direction of Android in the US whether or not they end up shipping phones there:

“US consumers will likely wind up getting a major dose of Xiaomi in how its Android phones are designed and how they work, even if they aren’t Xiaomi phones themselves. “

Moto G 2014

  • CAT Android phones feature prominently in this review of rugged smartphones:

Google and Android

  • Facebook have open-sourced Stetho, an internal Android debugging platform built on top of Chrome developer tools.  It should be of interest to mobile developers building connected apps on Android:

“Stetho does offer a couple of handy alternatives to the standard profilers. The first is a network inspector capable of offering image previews, JSON response helpers, and loading timelines. This helps developers find inefficiencies in their apps while pulling data from the Internet, just like they would with any web application. The other major feature is an interactive database front-end with the ability to read and write values to SQLite databases located in an app’s private storage.”

inspector-sqlite

Apps and Services

slack

instagram-30m-chart1

  • Nokia Radio Cloud has been unveiled by the company as a solution that virtualises their Radio Access Network (RAN) with the aim of distributing network capacity more efficiently.  The use of the term cloud appears aimed at telcos looking to “go digital”:

“The radio cloud is built on three layers, where the radio elements and antennas are on the first layer and connect to the second. The second layer then combines relevant cells into one cloud solution enabling traffic scheduling and coordination. The final layer contains the radio elements and/or base-stations connected to a centralised data centre, which is used to increase capacity temporarily.”

“Many operators are aware of the pressing need to transform themselves into digital telcos, but few are aware of the steps required to get there.  Only a holistic approach that reinvents the telco operating model can ensure operators avoid major business model disruptions and realize their digital telco vision.”

Asia

  • This very useful primer into Chinese web design trends highlights key developments in a rapidly evolving scene and also showcases what the author considers to be some of the leading Chinese agency-designed web sites.  Perhaps the most interesting observation concerns the rise of so-called mobile-only “Light Apps” frequently tied into WeChat:

“In 2015, Chinese companies are building light apps, or qing ying yong (轻应用). Light apps are one-off, zero-download, hyper-targeted mini-sites that are typically built (and often animated) with HTML5. Oh sure, they do have a dedicated URL and they’ll load in a desktop browser if you absolutely insist, but they’ll probably look awful on a desktop screen. In some cases, they don’t even function at all after the page loads, given that some of them require swipes to navigate. … WeChat is so popular that light apps are nearly always optimized specifically for WeChat’s native browser, and “Share to WeChat Moments” is by and large the only call to action in a light app’s interface.”

05-lenovo-light-app-opt-small

  • Interesting NYT insight into Chinese smartphone OEM Smartisan highlights a fiercely competitive world in which founder personality is a key factor in building brand awareness and loyalty:

“To compete with already established giants like Huawei and Lenovo, companies like Smartisan and Meizu use social media, the fame of their founders and just about any other Internet trick they can think of to build a name for themselves.”

Security

“The Mobile Handset Exploitation Team (MHET), whose existence has never before been disclosed, was formed in April 2010 to target vulnerabilities in cellphones. One of its main missions was to covertly penetrate computer networks of corporations that manufacture SIM cards, as well as those of wireless network providers. “

  • In contrast, those developing key open source security software outside of corporate patronage often face a precarious financial existence.   This recent article profiling Werner Koch, the author of GPG revealed his scant reward for a virtuous approach though happily, he did start receiving financial support from the Linux Foundation, Stripe and Facebook among others following its publication:

“Like many people who build security software, Koch believes that offering the underlying software code for free is the best way to demonstrate that there are no hidden backdoors in it giving access to spy agencies or others. However, this means that many important computer security tools are built and maintained by volunteers. … He says he’s made about $25,000 per year since 2001 — a fraction of what he could earn in private industry.”

Data Science

  • Evidence from the recent O’Reilly Strata conference in the US suggests Apache’s Spark in-memory data processing platform “looks like the future of big data” and will eventually overtake Hadoop as the go-to platform in that regard.  Although immature, Spark has an sweepingly ambitious architectural scope encompassing real time streaming, machine learning, visualisation and a programming model accessible via Scala, Python and R.

The Spark platform, minus the Tachyon file system and some younger related projects.

  • Microsoft too are accelerating their efforts to present a broad unified big data proposition built around their Azure cloud that is accessible to developers more familiar with Linux.  As such Microsoft will be also leveraging Linux to support their Hadoop implementation and indeed going further by:

“making its Azure ML machine learning service widely available now with new support for Python as well as the already-planned support for the popular R language.”

  • This amusing and insightful post outlines how the Priceonomics team built a tool to determine the “most interesting facts in the world” as measured by weighting responses to items posted in the Today I Learned (TIL) subreddit.  The single most interesting fact concerned Albert Einstein apologising for being someone else that was often mistaken for him:

“So our data crawling team built a simple algorithm that would crawl through the Today I Learned subreddit and count how many times a given article was submitted. Then, of all the times it was submitted, the crawler would choose the title that garnered the most upvotes (and therefore identified the most interesting fact in an already exceptionally interesting article).”

Wearables and the Internet of Things

  • The LG Urbane is a luxury Android Wear smartwatch proposition bought out just in time for MWC:

  • The Neptune Duo is a Kickstarter attempting to flip the master-slave relationship between smartphones and wearables.  It’s been initiated by the same Neptune that developed the chunky Pine smartwatch and comprises a wearable “Hub” and a phone “Pocket”:

“The Neptune Hub is a chunky wristband that actually houses all of the technology you’d normally find in a smartphone, and it’s accompanied by the Neptune Pocket, a 5-inch screen that just serves as a wireless dummy display for the Hub. “

  • Engadget weren’t overly impressed suggesting the product “reeks of a device built by geeks for geeks, but with little thought about how normal people actually use technology“:

“technology that can generate physical keys from a flat surface on demand, giving you a smooth, unbroken surface for general touch-based interaction, and a physical keyboard when you need one. The company’s innovation uses a microfluidic panel to achieve its magic, routing liquid through invisible channels to expand specific areas of the top layer of a touch panel, producing protrusions and bumps where previously there were none.”

phorm3

Raspberry Pi

IMG_8883s

Work and Culture

  • The author of Creators Code has tried to distill its lessons into a LinkedIn post.  In it she suggests that what Elon Musk, Jack Ma and other top business luminaries have in common is “entrepreneurial alertness“.  Namely the ability to “see what others miss” and exploit their clear understanding of gaps in current opportunities to build genuinely new business models from scratch. They clearly know a thing or two about how to build an exponential organisation too.
  • Hard to know quite what to make of this Telegraph article outlining “seven ways to appear more intelligent that other people“. The emphasis is on the perception rather than reality.  Turns out that being a self-effacing, quiet yet confident skeptic with a middle initial, smart friends and nocturnal habits should get you there.

Psychogeography

  • The extraordinary singular vision of James Brindle is highlighted in the latest episode of The Nor, “an investigation into paranoia, electromagnetism, and infrastructure”.  Among other things it explores the physical impact of High Frequency Trading on the built environment from Slough to the City in the form of mushrooming data centres and microwave dishes, the totemic enablers of the exponential.  The essay reads like an update to Iain Sinclair’s London Orbital twelve years on and likewise channels the dystopian vision of JG Ballard:

“Numbers, numbers, numbers. It’s grim out here. The road is cracked, the hedgerows are filled with rubbish thrown from car windows, I am accosted again by private security guards who don’t want me looking at their building, who threaten to call the police when I object to their request to run background checks on me. But the sun is shining; it is a beautiful winter’s afternoon. If you want a mirror city, this is it right here: billions upon billions of pounds being beamed across the rooftops of council blocks and crumbling hospitals, the haunts of pirate radio DJs now the domain of private hedge funds, and the once buzzing city centre finance sector displaced to silent halls on ring roads, staffed only by angry men in high-vis tabards.”

Week 9

$
0
0

Pick of the Week – Apple Car

More information emerged this week about Apple’s apparent electric car ambitions from the team assembled to build it to conjecture about the downstream supply chain implications.

Some commentators speculated on the likely impact to auto industry incumbents like Toyota, VW and Ford.  They would likely be first in line for the disruptive consequences that would ensue from an Apple entry into automotive rather as Nokia did before them:

“Even a modest Apple incursion into the automotive industry would likely prompt an entrepreneurial explosion of innovation—and innovative—partnerships. To what extent might an automotive counterpart of “apps” and the “app store” generate new automotive expectations and value?”

The Telegraph highlighted a suggestion from Tim Cook of an ambitious cross-product play centred around Apple Watch disrupting both your wallet and your car keys.  The article was notable for providing a glimpse into the genuine admiration and affection Apple leadership is held within the company in the description of a flying visit made by Cook to the London Covent Garden Apple Store.   The accompanying photo conveyed signs of almost religious asbsorption:

“I have witnessed many shop,office and factory visits before, with bosses being received in a variety of ways. In some cases, they were welcome but often they were met with indifference or, of course, outright hostility.

But the reaction at the Covent Garden store was off the charts: the staff gasped, and then burst into spontaneous, loud applause as soon as they spotted Cook, who walked in behind them.”

Devices and Manufacturers

“unlike smartphones which become obsolete within two or three years, Runcible’s parts can be removed, repaired and upgraded, enabling the device to be kept for decades.”

The world’s first anti-smartphone ‘Monohm Runcible’ is here

  • In terms of market, Runcible seems perfect for the sort of hipsters Nathan Barley parodied particularly the chap spotted in Dalston on a penny farthing:

“Runcible is banking on its retro appeal and is serious about bringing pocket watch designs into the 21st century. The gadget can be attached to a chain and and can even support a third-party clasp cover to flick open when you want to check the time, just like your great-grandfather probably did.”

  • Samsung may have announced the Galaxy S6 with its curved screen but they probably couldn’t design or release a product quite as radical Runcible. FastCompany help explain why in an article that implicates both ingrained company culture and the intransigent “Steve Jobs syndrome” exhibited by the Western design firms they employ:

“Samsung’s collectivist corporate structure won’t gamble $10million to bring a bold, unproven product to market – especially one that hinges on some holistic Apple-like experience – when the status quo of iterating old products and just tossing some new advertisable features into a sea of SKUs has a more predictable return.  And that phenomenon is undoubtedly holding the company back.”

“The bigger picture seems to be that Android handset makers are in something of a race to the bottom at this point: the markets that are driving growth in smartphone adoption these days are emerging economies, where consumers are price sensitive. That’s leading to the production of a number of models that are pushing down the average sale price for devices, which long ago dropped below the $100 mark.”

Screen Shot 2015-02-26 at 14.01.24

  • Arstechnica’s review of the Xiaomi Note is full of praise for the $370 smartphone taking aim at the iPhone6 Plus.   However, they emphasised how Xiaomi’s success in China operating outside of IP regulation will leave it vulnerable to litigation if it tries to sell its products in Western markets:

“If Xiaomi ever wants to expand to countries with strong IP laws, it will need to audit its software and change any features that could be infringing. In an IP landscape where nearly every smartphone company is suing every other company, lawsuits for Xiaomi currently seem inevitable if it starts selling smartphones in the West.”

  • The article outlines evolution of Xiaomi products from Mi1 through to M4 with the Mi Note at the end:

      1. Sony Pictures Entertainment
      2. PlayStation
      3. Selling image sensors to Apple for the iPhone.

“With nearly two-thirds of replaced smartphones being reused, continued demand for high-end used devices will increasingly impact primary-unit sales, and motivate phone providers to look into the secondhand market. In North America and Western Europe, the market for refurbished phones is forecast to be worth around $3 billion in 2015 and growing to $5 billion in 2017. Many users are attracted to used high-end devices that they would not have been able to purchase at the original selling price.”

Google and Android

“there will be two separate environments (personal and work) at the OS level for Android 5.0 (Lollipop) and beyond. But unlike BlackBerry’s secure work environment which has you switch back and forth between profiles, the Android for Work profile is integrated within a user’s personal profile.”

  • Google’s approach centres around the provision of specific dedicated Android For Work apps that are denoted with an orange symbol.  These apps can be locked down and controlled by corporate IT.  It’ll be interesting to see if this development changes the perception around Android as a genuine enterprise platform:

gfw device ui 2x 220x418 Google launches Android for Work as it tries to integrate itself deeper into the business world

  • The reason for the strange glitches many of us experienced with GMail addressee autocomplete last week can now be explained.

Apps and Services

“Magic charges a fee on top of the cost of the products or services you’re requesting, essentially acting as a middleman for services like Instacart or Seamless that are already acting as a middleman. That means you’re paying two premiums: one for the convenience of simplifying the process to just a text message and one premium if Magic ends up using an existing delivery service.”

Magic delivery concierge app

  • Talking of magic, PhotoMath is a great example of the current state of smartphone tech.  It combines real time text recognition with an equation-solving engine capable of displaying all the working to get to a solution for quadratic equations.  And it’s now available on Android as well as iOS:

  • The m-payments landscape remains hard to parse but seems likely to be centre stage in 2015.  Samsung’s recent acquisition of LoopPay provides them with technology that already works with 90% of existing EPOS terminals in the US.  However, it will only work on Samsung devices.  Google’s acquisition of “exciting technology” from carrier-led mobile payments consortium Softcard should significantly boost its ambitions with Google Wallet across all GMS-capable Android devices.  Both companies have Apple Pay in their sights.

LoopPay has been acquired by Samsung as it gears up to compete with Apple Pay. googlewallet

  • Silent Circle who are responsible for Silent Phone and Silent Text announced they are acquiring Geeksphone, makers of the Blackphone which by all accounts has done rather well since its debut last year.  The two companies were close partners setting up a joint venture called SGP to support the development of Blackphone and PrivatOS, the secure Android fork underpinning it.  The move puts Silent Circle in the unique position of being a leading security software provider with their own unique range of smartphone models including at least one to be launched at MWC.  It appears that they have also just secured an additional $50million in funding.

phone_flat_1

Asia

“When the host says it’s time, users of WeChat in China can shake their smartphones vigorously to be in with a chance of winning a portion of the cash available.”

  • Ford is seeking to integrate WeChat into the IVI of its cars in China.  According to Reuters it wants Tencent “to tailor its popular chatting app for the firm’s cars in China, as automakers in the world’s largest market vie for drivers that care about high-tech features as much as engine size.”
  • In a clear sign of the potential India represents, Google’s investment arm, Google Capital, is setting up a division in Indiathe first such expansion outside the U.S.

Big Data

  • Matt Asay reckons that the recently announced Open Data Platform (ODP) to unite behind a common Hadoop distribution for big data is doomed to fail like United Linux before it.   First of all “unlike the true Apache Hadoop standard, ODP “is clearly for vendors, by vendors“”.  Secondly, in spite of lots of hype, “adoption of Hadoop is not rising quite as dramatically as some might believe.”

Hadoop market

Artificial Intelligence.  And Hollywood

  • Viv, the AI technology built by the team that developed Siri has secured Series B funding of $12.5 billion.   It provides further validation for an approach that differs from the route taken by Apple in connecting with a much greater range of sources of information:

“Viv is different. It can parse natural language and complex queries, linking different third-party sources of information together in order to answer the query at hand. And it does so quickly, and in a way that will make it an ideal user interface for the coming Internet of Things — that is, the networked, everyday objects that we’ll interact with using voice commands.”

Screen Shot 2015-02-20 at 11.29.57 AM

  • Films about Alan Turing and Stephen Hawking were in direct competition at the Oscars this year in the form of The Imitation Game and The Theory of Everything.   Truphone CTO James Tagg used that as his starting point in comparing the two great scientists with the most striking difference being their attitudes towards Artificial Intelligence.  Turing of course was convinced of the long-term potential of AI.  Hawking is far more pessimistic.

“Unless you have a job where you have to do what somebody else tells you to do, then the only “person” qualified to be your boss would be a computer that was programmed especially for you, that would take into consideration all of your finances, prejudices, quirks, idea potential, temper tantrums, talents, personality conflicts, growth rate desired, amount and nature of competition, what you’ll eat for breakfast on the day you have to fulfill a contract, who you’re jealous of, etc.”

  • Warhol didn’t account for “algorithm aversion” which according to this HBR article arises when humans witness AI systems making mistakes inevitably resulting in existential distrust:

“people are even less trusting of algorithms if they’ve seen them fail, even a little. And they’re harder on algorithms in this way than they are on other people. To err is human, but when an algorithm makes a mistake we’re not likely to trust it again.”

Smart Watches and Wearables

“With LTE cellular access, the watch will function just like a phone. Beyond making and receiving calls and sending text messages, LG said the device will be able to send walkie-talkie-style short voice messages.  There is also support for NFC which will allow watch owners to use payment services directly from their wrist.”

  • Huawei too have launched a stylish premium quality smartwatch, the unimaginatively named Huawei Watch.  Unlike the LG Urbane it’s an Android Wear proposition which TechCrunch reckon is the most stylish smartwatch produced by any OEM to date:

  • Pebble Time is Pebble’s colour e-paper smartwatch proposition. It’s blew its funding target in just 17 minutes as befits the poster boy for Kickstarter.  Pebble Time should be released in May 2015 at a retail price of $179 with some nice touches including a PVD bezel, a range of colorful straps and an engraving option for Kickstarter patrons.

  • Jawbone on the other hand appear to be in some financial trouble and may be seeking investment from the likes of Google to keep them afloat.  Their much-anticipated UP3 smart band has been beset by delays and it now appears that their ODM Flextronics is suing them for non-payment.

Jawbone Up 3 Move fitness band wearable

embedded inside Swiss-made cases and powers the hour and minute hands as well as, in the current version, a subdial that tracks your activity …  essentially turning a fancy timepiece into a sort of mechanical Fitbit.”

Swatch's Touchscreen Watch Gets Smarter With Fitness Tracking Features

  • The tech world is “infatuated with fashion” in large part because the fashion industry knows how to market and sell form over function.  This skeptical post queries whether the likes of Apple and Google in particular will be able to play by the same “obnoxious” rules.  We’ll find out soon enough in Apple’s case with the Gold Edition Watch:

“Selling function isn’t something the fashion industry typically does. It sells a story, an identity, a new look. … It’s pure arrogance for Silicon Valley to imagine that it can make wearables cool by hiring a few fashion people, putting the product on a runway, or throwing money at “collaborations” with brands.”

Software

swift

  • This Python developer invoked the “easier to ask for forgiveness than permission” (EAFP) paradigm a little too literally and met with an embarrassing comeuppance when the man he swore at on the Tube on the way to a job interview turned out to be his interviewer.  He didn’t get the gig.

Space

Pia19185_ip

  • It seems eerily redolent of unknown alien intelligence from a Star Trek episode. Doubtless Spock would be able to provide a rational explanation. A fixture for those who grew up in the late 20th century, the actor who defined the role, Leonard Nimoy, passed on to the void this week but will hold a special place in the memory for many:

“for the millions of people, like myself, that grew up with Leonard Nimoy, those are the memories that we will carry with us throughout our lives.”

  • He remained class act down to his final tweet:

Work and Culture

  • This correspondent suggests that the Internet hasn’t fundamentally transformed employee productivity but that robots will because their rise will create “new industries and products”:

A group of programmable humanoid Nao robots, developed by a French company Aldebaran Robotics, perform dance inside the France Pavilion at the Shanghai Expo

“Tech is becoming the finishing school and springboard for the upper-middle-class, the way law and finance were a decade ago. Now that the tech industry is cool, the pretty people are taking over, flooding out of top-tier universities with MBAs and social graces and carefully coiffed hair, shouldering the misfits and weirdos out of the way.”

  • The corporate co-option of technology is something that Dan Gillmor bemoans in this Medium post.  In it he asserts that he is saying goodbye to Apple, Google and Microsoft and shall henceforth use only community-based platforms such as GNU/Linux and CyanogenMod.  Presumably as long as they don’t get co-opted too as is inevitably the case:

“I keep hoping more hardware vendors will see the benefit of helping their customers free themselves of proprietary control. This is why I was so glad to see Dell, a company once joined at the hip with Microsoft, offer a Linux laptop. If the smaller players in the industry don’t themselves enjoy being pawns of software companies and mobile carriers, they have options, too. They can help us make better choices.”

  • Millennials have stopped job hopping since the downturn and this LinkedIn post considers that “a bad thing” because it means the corresponding range of job opportunities needed to support such job hopping aren’t out there.

W150220_WILLYERD_HOWOFTENEMPLOYEES (1)

“The key takeaway? Red, pink, and purple seem to promote repinning, while green, blue, black, and yellow seem to suppress it.”

The original image is in the middle. At left, white-balanced as if the dress is white-gold. At right, white-balanced to blue-black.

  • The New Scientist provided a good explanation why both answers are right because what you see depends on how you “discount for the colour of the illuminant“.  It’s a reminder of how surprisingly subjective a quality colour actually is:

Week 10

$
0
0

Pick of the Week – Disintermediation

Mobile application-mediated service delivery is arguably the most powerful agent of disintermediation the world has ever seen and has been the key enabler giving rise to the Exponential Organisations discussed a couple of weeks ago.  TechCrunch this week published a post outlining why user experience is central to the success of the disrupters.  With billions at stake the interface has now become the primary battlefield for everyone in mobile:

“The Internet is the most powerful mechanism we can imagine to match perfectly individuals that need something, and people with something to offer.  …   In this age, the customer interface is everything [the] indescribably thin layers that sit on top of vast supply systems ( where the costs are) and interface with a huge number of people ( where the money is). There is no better business to be in.  The value is in the software interface, not the products. .. The Internet age means building things is nothing other than code. We’re going to see a non-stop battle to leap ahead of each other.  .. having icons on the homepage is the most valuable real estate in the world, and trust is the most important asset. If you have that, you’ve a license to print money until someone pushes you out of the way.”

The competition is so intense that nobody, least of all the primary mobile platform providers, can rest easy.  Google in particular are at real risk of being disintermediated in turn by Chinese OEMs forking Android to build a third platform.  This is the key reason why we should care about Xiaomi, an otherwise “faraway” low-end Chinese manufacturer:

“low-margin companies using commodity components and a commodity OS, yet achieving differentiation in design, software and services. Looking at China alway challenges your assumptions about what’s inevitable in technology.”

Apple for now is apparently immune, once again atop the mobile world and the centre of attention despite having no overt presence at MWC, the industry’s annual get-together.  Even so, Apple need their Watch and its multiple variants to open a new front in the interface war and fuel their continuing success.  With launch due this week they won’t want attention focussed on all the legitimate reasons you might want to avoid buying v1.0 of Apple Watch.  Many will understand the rationale yet be unable to resist the siren lure walking past an Apple Store in the coming weeks.

5 Reasons NOT To Buy An Apple Watch!

Table 1

Worldwide Smartphone Sales to End Users by Vendor in 4Q14 (Thousands of Units)

Company

4Q14

Units

4Q14 Market Share (%)

4Q13

Units

4Q13 Market Share (%)

Apple

74,832

20.4

50,224

17.8

Samsung

73,032

19.9

83,317

29.5

Lenovo*

24,300

6.6

16,465

5.8

Huawei

21,038

5.7

16,057

5.7

Xiaomi

18,582

5.1

5,598

2.0

Others

155,701.6

42.4

111,204.3

39.3

Total

367,484.5

100.0

282,866.2

100.0

Source: Gartner (March 2015)

MWC, Devices and Manufacturers

“here in Barcelona, the Apple shadow was everywhere. The company is, by all accounts, a powerful moneymaking machine and, on top of that, the devices and brand are widely adored. That is a pretty steep hill for anyone else to climb.”

MWC 2015: virtual reality surprises, curved screens and connecting the world

“The Galaxy S6 has a flat display, metal frame, and glass front and back panels. Its counterpart, the S6 Edge, has all of the above, but throws in a curve to the sides of its front and rear glass, giving it a distinctive character.

Both of the new phones are made entirely of metal and glass: the cheesy and cheap-feeling plastics that have dominated Samsung’s products for years are no where to be found. This change can’t be overstated: Samsung finally has made flagship products that look and feel like they are worth the premium price tag they command.

Samsung’s lead designer on the project says the company spared no expense when it came to materials that they could use in the new phones.”

  • Both models will be available next month and will compete for column inches with Apple products.  How the S6 Edge in particular fares will be crucial to Samsung’s turnaround plans.   They’re attempting to redefine the high end again after having phenomenal success creating and building successfully on it from 2009:

“Samsung created a luxury sub-brand for its Android phones moving forward, the Lexus to its Toyota.”

  • Since 2013, however, Samsung have stumbled with both the S4 and then the S5 struggling in the face of serious competition from Chinese vendors like Xiaomi.  Samsung’s decline has been as sudden as their rise and it’s entirely possible that the company has already lost the smartphone war in spite of the generally favorable reception for the S6.  It’s questionable if even the curved wow factor of the Edge will be enough.  As Benedict Evans put it:

“the Edge variant, in which the screen bends around the end of the phone at each side, is probably an acquired taste – there’s no fundamental reason for it beyond Samsung’s screen division being able to do it.”

  • One possible future alternative direction for Samsung would be to licence its chipset and software technology to other smartphone OEMs and try to usurp Qualcomm rather than Apple:

“Unless Samsung has a special trick up its sleeve on the software side, it’s unlikely that its new phones will be enough to justify the extra cost over similar Android devices. And if that happens, Samsung is almost certainly up for another messy year. The glow surrounding Samsung’s smartphone business has almost certainly faded for good. Time to find something new.”

  • Engadget’s top ten smartphones from MWC included the Samsung S6 and S6 Edge, Runcible, the Blackberry Leap, and the HTC One M9 “a phone that’s really lovely, but it’s maybe not as exciting as it could have been“.

  • TNW’s review of the HTC One M9 provides some further insight into the hardware and HTC’s new Sense 7 UI.  The product will switch from using Beats to Dolby for audio credibility and the product emphasises a premium differentiated look with software personalisation a key theme.

“Put bluntly – it felt real. The whole experience was oddly breathtaking. I lost awareness of the headset, which fitted comfortably, and the plastic of the controllers in my palms and for around 15 minutes really felt as though I was in another world.

It sounds ridiculous, but I was grinning like a lunatic the whole time. I can’t urge you strongly enough to try it for yourself, and HTC’s vision for the headset extends beyond gaming; incorporating shopping, travelling and even speaking with friends. And it deserves to be enormously successful.”

“The new smartphone features built-in encryption, malware protection and remote backup to ease administrator’s security concerns, and has access to both the BlackBerry World and Amazon Appstore for applications. BlackBerry claims that even with ‘heavy use’, the Leaps’ battery will last for a full day.”

Leap_Grey_Front

  • Another competitor vying for the same lucrative enterprise security market also launched at MWC is Silent Circle’s Blackphone 2.  If anything, it has an even more credible enterprise security story built on an secure fork of Android which now supports application containers:

“Blackphone hardware runs a hardened Android fork, called PrivatOS, so it’s also compatible with Android apps.  Last December SGP announced it would be adding in its own app store in an update due early this year — along with a feature called ‘Spaces’ that will let users segment activity on their device by creating multiple secure spaces for particular apps, accounts or data to help balance security and usability.”

“We tried it a handful of times and it worked as promised on every occasion. It’s not quite as fast as using Touch ID, but a huge advantage for smartphone makers is that it doesn’t require extra hardware. “

  • Amongst the other new smartphone propositions that stood out at MWC were French manufacturer Wiko who have captured 10% of the French market:

Microsoft

  • They’re also apparently explicitly collaborating with Google on Angular 2 which would have been inconceivable under Ballmer.   Angular 2 is Google’s JavaScript web app framework and it appears it will converge with Microsoft’s own JavaScript extension project, TypeScript.

Google and Android

  • Google provided a developer-oriented breakdown of some key upcoming improvements in Google Play Services 7.0 including the introduction of a Places API which “makes it easy to get details from Google’s database of places and businesses“.   Places could offer smaller development outfits a way of hosting and accessing content without needing a CMS albeit one that only works outside China:

“In September, we announced that all new Android Lollipop devices would be encrypted by default. Due to performance issues on some Android partner devices we are not yet at encryption by default on every new Lollipop device. That said, our new Nexus devices are encrypted by default and Android users (Jelly Bean and above) have the option to encrypt the data on their devices in Settings —> Security — >Encryption. We remain firmly committed to encryption because it helps keep users safe and secure on the web.”

  • Cyanogen are rebranding themselves in an apparent attempt to move away from an explicit Android connection and position themselves as a third OS option.  As part of that move, they’re also partnering with Qualcomm to ensure the platform runs on QRD (Qualcomm reference devices):

“The Cyanogen partnership will provide software features and UI elements for devices running Qualcomm Snapdragon 200, 400, and 600 processors, which are used in low-end to mid-range phones.

Mobile Payments

  • Sundar Pichai announced an Apple Pay competitor called “Android Pay” at a Google keynote at MWC.  Specific details were not provided.  It seems the proposition “would “start with NFC” and eventually accommodate biometric sensors as well” and that it will be “separate” to Google Wallet though both will build on the same underlying tech.   Android Pay is positioned as a platform play accessible on any compatible Android device through Google controlled developer APIs.  Apple Pay by contrast will only be available on Apple products.
  • Stratos is an interesting new entrant in the mobile payments space, a “connected Bluetooth payment card” that should work like a conventional credit card with existing EPOS infrastructure. Explicitly designed to “to replace every single plastic card in your wallet“, it uses a Bluetooth connection to the owners iPhone to augment the payment experience to cover an apparently expandable range of use cases:

“Using the companion mobile app, you can switch between cards and track your spending. The company is working to add further features like location based recommendations based on your spending history, virtual card downloads (think gift cards or pre-paid cards) and more.

The amazing thing about Stratos is that the company says you can load basically anything that features a magnetic strip onto it like credit, debit, loyalty, membership or even gift cards.”

“To protect your information, the app can tell when the card is out of Bluetooth range, and after a pre-determined time — anywhere from 15 minutes to 24 hours — it can disable the card altogether.”

Stratos card hands-on

  • In another validation of the smartwatch as a payment vehicle, MasterCard outed their Android Wear payment app at MWC:

Apps and Services

“Tying pricing to a user’s age won’t end well for Tinder, who will probably get backlash from users who feel they shouldn’t be penalized financially for their age. Adding paid tiers is a shameless and clear indication that Tinder does not want older users and will make them pay extra to get the best version of the service.”

Tinder screenshots

  • Yet more evidence from HBR that engagement with Social Media sites may help brand visibility and relevancy but doesn’t actually help boost company profits:

“62% of U.S. adults who use social media say these sites have no influence on their purchasing decisions and only 5% say they have a great deal of influence.”

“Firefox Hello allows users to have Skype-style video conversations but there is no need to sign up or login, which according to White gives the kind of privacy most consumers like. The service works with the user opening a session within the Firefox browser, and sending a link by email to whomsoever they wish to speak.”

Asia

Top 10 best Chinese smartphone brands

 

Databases

  • Matt Asay outlined a collection of field data that supports the assertion that NoSQL database technology roll-outs led by MongoDB and Cassandra are eating into the dominant relational database market.  The data includes an analysis of “media mentions” illustrating a clear industry interest in MongoDB:

“What is most interesting in the media mentions data, however, is the trajectory: all NoSQLs are growing, and some quite substantially, and mostly at MySQL’s expense “

Figure A

Pebble Watch

  • Pebble’s next big thing, a colour version of their popular watch, has now become the most funded Kickstarter project ever.  It beat its old Kickstarter record within 2 days and now stands at nearly $17million pledged.  It’s an incredible achievement and one which ought to serve as a huge wake up call for any in the industry still complacent about the viability of crowdfunded propositions.
  • Pebble Time also has an all-metal premium variant called Pebble Time Steel:

“Time is suddenly no longer just another smartwatch in Pebble’s lineup; it’s now a full wearable platform that can potentially integrate with countless other devices in your life — assuming Pebble can get other hardware developers on board.”

pebble time wm 1

“The connection can send and receive power and data to the device which opens the way for smartstraps. They’ll be able to replace the normal watch band and offer extra features via dedicated APIs. … The APIs will allow developers to read and write to the port, have the device detect when a smartstrap is attached, and automatically launch apps and functions. … Developers will be able to create accessories that extend the Pebble Time’s functionality beyond the sensors built in to its main body. The company uses the examples of adding GPS, a heart rate monitor, NFC and greater battery life but the potential is huge.”

strap components 520x509 Pebbles smartstraps are a big deal and bad news for Apple

Science

  • They also released a short YouTube video outlining albeit at a pretty high level how the photograph was taken:

Software Development

  • Quora questioner asks “how many lines of code is Facebook” and the answer comes back from an employee that it’s a staggering 62 million SLOC for the “main site” without the backend – from memory that’s about double the eventual size of Symbian OS:

The Android codebase is over 4 million and the main site without the backend code is 62 million.

Clarification: The new count includes the entire git repository: data, binaries, third-party and all. … The number is from running:

git ls-files | xargs cat | wc -l
  • Crossy Road is the latest cult retro gaming phenomenon, a Frogger throwback with fun basic graphics and unambiguous monetization credentials.  This post provides a great profile of the game, how it came to life and how much it has made just three months after launch:

“90 days after its release, Crossy Roads combination of solid gameplay, unobtrusive in-app purchases, and optional in-app ads powered by the Unity engine, has earned $10 million from 50 million downloads.”

And this one.

Cube Drone by Curtis Lassam.

  • It also highlights how Angular.js seemed to rise to dominance in last couple of years though whether that remains the case given the response to Angular 2, remains to be seen:

Work and Culture

  • It’s fairly commonplace these days to read commentators describing how we are on the cusp of the third industrial revolution.  This post sourced from Forbes addresses the topic from an interesting new perspective, namely energy and finance.   It suggests that what will really characterise the third industrial revolution is the potent combination of crowdfunding and solar energy:

“In the First Industrial Revolution, which came out of England, the new energy source was the change from wood power to coal power.  The new financial system was the London stock market. … In the Second Industrial Revolution, driven by technological advances in the UK, USA and Germany, oil replaced coal, again much more energy-dense.   And the new financial breakthrough was the limited liability corporation which reduced risk to individuals engaged in entrepreneurial activities. …  The new energy source looks very likely to be predominantly solar power. The new financial system is less advanced, but internet-driven breakthroughs such as crowdfunding and peer-to-peer finance are indicating a democratisation of finance.”

“I will only hire someone to work directly for me if I would work for that person,” Mr Zuckerberg told the audience.

“It’s a pretty good test and I think this rule has served me well.”

 

Week 20

$
0
0

The rise of no-stack solopreneurship

TechCrunch asked this week if it is possible to predict which startups will turn out to be unicorns and offered this reasonable if, ironically, somewhat predictable guidance:

“to predict the unicorns, we need to make some bets on what trends will take us by surprise over the next decade and what will get real big real fast.”

Rather than any specific technology trend such as 3D printing, machine learning or robotics, they chose to focus on the rise of the “solopreneur” as their primary megatrend:

never before in history has it been so easy for anyone to run a seemingly huge looking company or reliably valuable service just from their home or by themselves. Even a non-technical person can start their own company pretty easily and be selling their product, service or skill without investing thousands of dollars or months of risk and fit out.

HBR expanded on this theme and identified three other megatrends. They suggested that the extent to which you are exploiting these three can be used to predict the future of your business:

  • Machine Learning – will increasingly be incorporated into the fabric of business, driven in part by efforts by the likes of AWS to democratize access to the underlying technology.
  • New Urban Infrastructure – the importance of the city as the primary built environment for human habitation and interaction.
  • Scalable Niches – the shrink wrapping of critical business functionality behind web scale service APIs.

Of the three arguably the last has the most profound implications for solopreneurs everywhere seeking to build the next unicorn. “Scalable niches” enable small agile outfits to dramatically level the playing field against more slow-moving corporate behemoths by allowing them to more efficiently and rapidly ingest the innovation of others:

Companies like Twilio, Box, Zipments, Sendgrid, Stripe, BigML, and hundreds more have invested thousands of years’ worth of developer time in standardizing the basic services that cloud software vendors require. And what’s more, unlike a world of developing for client based systems, these vendors continued to invest an insane amount in simplifying integration, deployment, and the adoption of new innovation for their customers. The end result is now, when you build a new piece of software, you can concentrate on what’s truly critical to your differentiation.

It’s a theme echoed in this companion piece proclaiming the rise of the “no stack” startup:

Thanks to companies like Stripe and Twilio, which offer robust, well-documented APIs that are either free or affordable to get started with, developers have access to more core functionality than ever at their disposal. Infrastructure and back-end services that previously would have required a large investment of time and money to replicate are now available with integrations that can take as little as a few hours or days.

No stack adherents need to be acutely aware that their approach incurs platform risk.   Nevertheless, the combination of speed and global scale will prove irresistible to many and when considered in tandem with paradigm-shifting combinatorial advances across a range of technologies, there’s probably never been a better to time to be a tech solopreneur.

Manufacturers and Devices

“if big American corporations and European politicians had any imagination, they could probably engineer a bailout for the nearly bankrupt country on terms that would benefit everyone.  … Greece, for its part, would get debt relief, plus the companies’ European headquarters.”

  • According to a well-regarded analyst, Apple may extend their foray into super luxury tech products from the Apple Edition Gold Watch to a rose gold iPhone 6s variant later this year. However, it’s not entirely clear if the rose gold in question refers to simply a colour or the actual material.

On average, across Europe’s big five countries during the first quarter, 32.4% of Apple’s new customers switched to iOS from Android

the 20nm Helio X20 is actually more about its “Tri-Cluster” architecture that consists of two 2.5GHz Cortex-A72 cores, four 2GHz Cortex-A53 cores and four 1.4GHz Cortex-A53 cores.

“[YotaPhone] announced a revamped version of its second-generation handset that promises improved performance, significant software tweaks and a new black on white design.  The ‘limited edition’ white handset … introduces the second-generation of the company’s custom YETI UI for the ‘Always On’ rear display.”

YP_lollipop2

Microsoft

  • TechCrunch consider the implications of Microsoft’s announcement that it is going to shift from a shrink-wrapped software approach with Windows 10 to a service approach with continuous delivery of features and fixes:

“Putting Windows into that bundle of services would be a strong move toward upping the overall value proposition of the whole. And moving to a more gradually iterative model would probably have benefits in terms of engineering resource allocation and keeping pace with the increasingly rapid adoption curves of new tech.”

  • Windows Phone remains squeezed between iOS and Android. Microsoft presumably hope that by renaming it “Windows 10 Mobile” some of the Windows 10 cross-platform hype will rub off.  However Charles Arthur of the Guardian argues that Microsoft’s announcement that Windows 10 will support iOS and Android apps won’t help prospects.  It’s a “white flag to rivals and a red flag to developers“:

Microsoft had to do something, and people who like clever technical solutions are delighted by this clever technical solution to the fact of developer indifference and incompatible software. But it doesn’t change the fundamental truth: Windows Phone (v10 or whatever) is too small to matter in platform terms on mobile.

Apps and Services

“All of us who passionately believe in the value of content, who grew up with newspapers and magazines as essential to our knowledge-base and worldview, and who understood the value and recall print being a superior advertising and circulation conduit for advertising and paid subscriptions, should be very concerned and dismayed by AOL’s sale to Verizon.”

  • AOL was once a giant in the OTT messaging space with AIM.  This excellent article provides lots of fascinating insights into the cat and mouse war of attrition that went on between the AOL AIM and Microsoft MSN Messenger engineering teams in the late 90s. They fought each other to a veritable standstill with a variety of devious hacks and countermeasures to retain market dominance including apparently the injection of buffer overrun exploits within protocol messages.
  • HBR on what the rise of OTT Messaging means for brands. Among other things it will force them to address some very difficult problems such as how to consolidate a variety of data sources (in many cases unstructured) to present a unified customer view and how to dismantle rigid cross-functional organisational silos.

Top Gear Live

Security

“app users would do well to unlink their credit card, then change the account password before relinking it. Or maybe just pay with cash next time.”

A paper coffee cup was left outside a Starbucks coffee house

Asia

“Both the brands were losing sales to online-only smartphone brands at a time when e-commerce already contributes 15% of smartphone sales in India and expected to double this year”

Software Development

  • Rust 1.0 has been released.  The language is touted as “a drop-in replacement for C” which “combines low-level control over performance with high-level convenience and safety guarantees.
fn main() {
    println!("Hello, world! Rust here.");
}

“Because requests excels at grabbing HTML/XML/JSON a tool like pyQuery can help coerce, analyze, and query those responses from a different angle. pyQuery behaves a lot like jQuery in the browser, allowing CSS-based selectors to find HTML elements within an xmltree.”

  • Good post from a US context on why learning to code at school is a vital responsibility of the educational establishment with much improvement required:

“creating a future army of coders is not the sole purpose of the classes. These kids are going to be surrounded by computers—in their pockets, in their offices, in their homes—for the rest of their lives.  The younger they learn how computers think, how to coax the machine into producing what they want—the earlier they learn that they have the power to do that—the better.”

Web, Cloud and Devops

1. Write the following to 'swapme.sh':
#!/bin/bash
file='/var/swap.1'
dd of=$file if=/dev/zero bs=1M count=1024
mkswap $file
swapon $file
fstab="$file swap swap defaults 0 0"
echo $fstab >> /etc/fstab

2. Run it and change permissions on swap:
$ chmod u+x swapme.sh
$ sudo ./swapme.sh
$ sudo chmod 600 /var/swap.1

3. Build matplotlib
$ sudo yum install libpng-devel
$ sudo yum install freetype-devel
$ pip install matplotlib
  • SplunkLive! took place in London this week.  It was even bigger than last year’s event, testament to the growing reach of Splunk as an operational intelligence platform.   A particular highlight was the keynote from PaddyPower CIO Fin Goulding on how they integrate Splunk within a broader DevOps culture at the company.  Another highlight was sitting through a hands-on class going through the Splunk Enterprise 6.2 feature set courtesy of some cloud based instances that disappeared at midnight.

“a set of practices, tools and policies that lead to improved quality and Automated Delivery (AD). In many ways, rapid and frequent deployment to production reduces risk, as any release contains fewer changes.”

“With this approach, trust is moved from the network level to the device level. Employees can only access corporate applications with a device that is procured and actively managed by the company. In this setup, Google requires a device inventory database that keeps track of computers and mobile devices issued to employees as well as changes made to those devices.”

If we have learned anything over the last 18 months from Sony, Target, Anthem, JPMorgan, et. al.; it’s that there’s no safety behind the firewall anymore.

Wearables and the Internet of Things

  • The LG Watch Urbane generated some interest when it was first announced earlier this year.   It’s an Android Wear accessory with premium aspirations.  This Engadget review suggests it falls short mainly on account of a “clumsy, ostentatious design“:

LG Watch Urbane review: a premium watch that falls short of greatness

The Activité is a touch pricey ($400), but it’s a fine looking watch that has an old-school feel while still offering functionality you’ve come to expect out of an activity tracker.

“the FR225 is basically a FR220 running watch with an optical HR sensor stuffed inside and then the Garmin Vivo lineup of activity tracking functionality (i.e. steps) added to it as well.”

  • The Jawbone Up3 has undergone a troubled gestation with many exasperated by a lack of communication from the company on various delays in getting the product to market.  Engadget finally secured a unit for review and found it a “feature-packed disappointment”.  They suggested it was too expensive for a wearable without a display:

OLYMPUS DIGITAL CAMERA

Machine Learning and Data Visualisation

“If you have ever built a classification model yourself, you know that you should carefully choose your model type based on your specific use case. What I found very powerful in AWS Machine Learning is that you don’t need to care about that, since it automatically trains and tests a lot of complex models, tuned with different parameters, so that the best one will be chosen for the final evaluation.”

  • This Tableau white paper outlines which graph you should use based on the input context and what you’re trying to cover. Gannt charts to bubble charts to column, bar, pie and scatter charts are all covered.
  • Another link to a post outlining the best chart to use to visualise your data:

  • FiftyThree’s popular award-winning Pencil and Paper calligraphy apps for the iPad have been enhanced with a set of tools called Think Kit aimed at disrupting Powerpoint.  Here’s a compelling video of Think Kit in action with Paper and Pencil:

Artificial Intelligence, Robots and Lego

  • This inside view on Google’s self-driving car program written by its director Chris Urmson, highlights their overarching focus on safety.   Progress to date has been impressive and has been centred on the continuous integration of telemetry from laser rangefinder, conventional radar and GPS sensors within the context of SLAM (simultaneous localisation and mapping).  Stats highlight the relative safety:

“Over the 6 years since we started the project, we’ve been involved in 11 minor accidents (light damage, no injuries) during those 1.7 million miles of autonomous and manual driving with our safety drivers behind the wheel, and not once was the self-driving car the cause of the accident.”

  • However it’s not just human error autonomous vehicles have to contend with.  Localisation requires comparison against a known map which is fine in sunny California but can present problems in climates subject to extreme weather changes.  The current state of the art can apparently cope with rain but not snow.

What changed is the amount of computing power we have at our disposal. We can now run these systems across dozens, hundreds, even thousands of high-powered processors.

Give the Wolfram Language Image Identify Project a picture, and it uses the language's ImageIdentify function to identify it

  • Quartz profile the Future of Humanity Institute in Oxford which is headed up by Nick Bostrom and focusses on the study of “existential risk”.   The scientists and researchers who work there are billed as “the people trying to prevent humanity from destroying itself“, a task sufficiently grand that it constitutes arguably a modern-day Manhattan Project.

Startups and Unicorns

  • How to build a unicorn from scratch and walk away with nothing. A cautionary tale of venture capital, dilution, ratchets and above all terms from an inside perspective.  The moral of the story is to focus on terms and not on inflated valuation.  The former will determine what share of proceeds a founder will be eligible for in different circumstances.
  • An Asymco post simply entitled “Unicornia” attempts to analyse the core characteristics of unicorn companies.   His main conclusion looking at a range of corporate indicators is that Unicorns should be “characterized as having negligible revenue or profit but significant growth.”  He also picks apart the commonly-held view that a fixed value per user can be used to drive overall market valuation of a Unicorn.  In reality this figure is subject to huge variation across different companies with Apple having the most valuable users by far:

Screen Shot 2015-05-15 at 5-15-2.47.16 PM

  • Interesting post from one of the founders of Gamevy on how a strict adherence to the Lean Startup mantra almost killed their company.   An important point the author makes is that the minimum viable product (MVP) is NOT necessarily the same as the minimum desirable product (MDP) and that the latter is more important for a startup because of the severity of the competition and the difficulty of breaking through.
  • James Altucher in TechCrunch suggests asking the following nine questions before accepting an offer of employment from a startup:
    1. Has the CEO built a business before?
    2. Does the company have good funding?
    3. Do you believe in the vision?
    4. Do you believe in their latest round valuation?
    5. What can you learn?
    6. How do the partners get along?
    7. What are the current demographic trends?
    8. When will they IPO?
    9. Do you see the path to profitability?

Work and Culture

“Delivering great customer experiences requires marketers to touch every aspect of an enterprise, from sales and products all the way fulfillment. This is unchartered territory for marketers which may explain why CEOs currently don’t see enough CMOs stepping out of the marketing bunker and putting on their business hat.”

“Where Mark Zuckerberg wants to help you share baby photos, Musk wants to … well … save the human race from self-imposed or accidental annihilation”

  • Internet forums and social media provide a fertile combination of anonymity and distance to allow trolling to thrive.  Being righteous about it won’t help:

“The internet will always reward extreme views with extreme attention. … Use your brain – but don’t let yourself be manipulated into wasting energy shoving back against something that’s just meant to get your goat.”

“Responding to a troll just gives them evidence of their success for others to enjoy, and powerful incentive to try it again to get a rise out of the next sucker and satiate their perverse desire for opinion bloodsport. Someone has to break the chain.”

Election

  • The Reinvigorated Programmer offers his assessment of why the UK election result went the way it did counter to all the polling. In short, many people upon being faced with a number of uninspiring alternatives chose the least worst one from the perspective of self-interest.  It doesn’t seem to be a recipe for inspiring politics.
  • This NYT opinion piece extends the theme of parochial self-interest further to a national level within Britain and suggests that the real “winners” from the election were actually those who champion the identity politics of England and Scotland.

Week 24

$
0
0

Code

Bloomberg devoted a whole issue to a single article written by Paul Ford attempting to convey the world of corporate software simply entitled Code.   The online version is an extraordinary pyrotechnic tour de force that comes with its own GitHub repository for the awesome interactive graphics like this simulation of logic gate operation:

CircuitryCode

It is a long read at 38,000 words but well worth the effort particularly if you work with software technology but don’t have a great deal of insight of what software is and how it works:

It takes something both very important and hard to understand, and makes it understandable to an audience of smart but non-expert readers. It does this incredibly well. It mostly feels like fun, not work.

Reading it made me realise that a lot of my career has been spent being “the man in the taupe blazer” or TMitTB as Ford calls him:

Here we go again. On the other side of your (well-organized) desk sits this guy in his mid-30s with a computer in his lap. He’s wearing a taupe blazer. He’s come to discuss spending large sums to create intangible abstractions on a “website re-architecture project.” He needs money, support for his team, new hires, external resources. It’s preordained that you’ll give these things to him, because the CEO signed off on the initiative—and yet should it all go pear-shaped, you will be responsible. Coders are insanely expensive, and projects that start with uncomfortably large budgets have an ugly tendency to grow from there.

Ford’s epic helps explain why software skills are in such high demand that they are eating the job market.   Coding is just one aspect of this phenomenon however.  If anything, data analytic capabilities are in even more unprecedented demand.  There has never been a better time to develop software and analytics skills:

“organizations that are leveraging analytics in their strategy have not only world-class data scientists – but they also support “analytical amateurs” and embed analytics throughout all levels of their organization and culture. For this reason, the need for analytics skills is exploding within a variety of employers, and analytics and data-related themes top many corporate strategy agendas.”

Apple

  • Apple’s annual Worldwide Developer Conference (WWDC) took place this week.    The most noteworthy announcement was the launch of the Apple Music streaming music service promising to “change the way that you experience music forever“.  Apple are promoting the primacy of human curation over algorithms with radio stations like the global Beats 1 where a “team of experts” led by ex-Radio 1 DJ Zane Lowe choose the playlist.  Priced at just $9.99 a month following a complimentary 3 months, it’s clearly taking aim at Spotify Premium.  And there’s a tie-in of course with Siri:

If you’re a big fan of using Siri to pick songs, you’ll be able to do that with Apple Music. The virtual assistant can pick tracks based on the year of release, artist or chart position for a given month and year (April 2012, for example).

The proactive assistant is a mix of Google Now and similar tools that aim to make the user’s life easier by trying to infer what they may want to do at a given time — but with a focus on privacy.

apple-wwdc-20150142

In Apple’s favour is the fact that contactless payment methods are well established in the UK, ironically aided by the likes of McDonald’s. … Hence, Apple may find that the UK is a far more receptive audience than in the USA, where the entry of Pay has had a relatively low impact.

  • Apple Pay is not entirely without risk.  Mobile security firm Wandera published details this week of a specific attack that harvests credit card details by spoofing the enrolment process. Whether this sort of issue makes any dent in consumer uptake remains to be seen however:

Using inexpensive (less than $100) and readily-available, portable equipment in a laboratory setting, Wandera’s security researchers have identified a social engineering method in which hackers can inject a fake captive portal page which pops up and imitates the Apple Pay enrolment process, prompting the user to enter their credit card details. These details, including the security code, can then be easily harvested and used for fraudulent purposes.

pay

Chinese luxury smart watches

Chinese luxury gift brands

  • This LinkedIn respondent explains how the luxury watch industry should respond.  In short, not by trying to ape Apple but rather in moving the debate to higher ground:

The key question for me is how the luxury watch market should respond to this challenge. I believe the single biggest risk that they face is creating products like the Apple watch. This will signal the beginning of the end. As a luxury watch maker they must uphold luxury values. They must bestow on their owner the sense of time and place that reflects new luxury values. To move away form this would be to move away from the thing that is most valuable. The thing Apple wished they had.

Devices and Manufacturers

  • HTC have dramatically cut their revenue forecast for Q2roughly 30% downward revision of the guidance it offered just a couple of months ago“.   It seems their Android smartphone flagship, the HTC One M9, hasn’t really taken off despite initial promising signs post MWC.   According to their CEO they need to diversify and “aggressively develop new business opportunities beyond smartphones” to differentiate themselves:

“When the One M9 was launched many commentators were underwhelmed by what they perceived as the lack of innovation in a smartphone that was very similar in design and spec to its predecessor. But HTC is hardly alone in this respect, with many of its competitors struggling to demonstrate meaningful innovation every year. Compounding the problem, market leaders Samsung and Apple are perceived to be offering something new and it’s likely that the Q2 smartphone numbers will show increased market share for both of them.”

  • Another compounding factor are persistent reports of overheating and poor camera experience on the One M9 which is built on the Qualcomm 8994 64-bit chipset.  HTC have finally this week pushed out a software update that sounds like it addresses some of those issues.

it’ll fit in your clothes and takes photos that’d make your phone cry. … Sony claims it’ll shoot near-broadcast-quality 4K video for five minutes at a time, and can record slow-motion video at up to 960 frames per second.

Google and Android

whichphone

three of China’s six major handset makers–ZTE, Huawei, Coolpad, Lenovo, Vivo and Oppo–received offers from Google, which would pay US$1 for each phone that is pre-installed with a version of Google Play.

Apps and Services

we can see that the average app loses 77% of its DAUs within the first 3 days after the install. Within 30 days, it’s lost 90% of DAUs. Within 90 days, it’s over 95%. Stunning. The other way to say this is that the average app mostly loses its entire userbase within a few months

retention_graph_average

  • Even best in class apps are not immune to the “most tragic curve in tech” though they plateau at a much higher percentage of active users.  Chen’s view is this is down to a relentless focus on first use experience and direct relevant content to ensure initial and ongoing attention:

android_retention

Duolingo says its free language learning tool currently has over 100 million users worldwide. More people are now using it to learn languages in the U.S. than in the entire public school system, the company claims.

  • Talkdesk is a good example of the kind of new enterprise proposition that has been enabled by cloud technology and they’ve just secured an additional $15million funding round. Their platform allows customers to spin up a scalable, virtual call centre via a web browser and configure it to connect to other popular enterprise SaaS business tools:

Companies use Talkdesk to create centers where customers can call into in order for assistance and feedback. Those calls are connected to internal company databases like Salesforce and Zendesk, which bring up all the relevant information about a caller in order to help a customer service representative better resolve the issue or route that customer to the best person.

Teens who don’t have a data plan that will allow them to text are using their iPods and iPads to message each other on a closed network within a 100-foot area within school limits.

  • Firechat offers similar functionality but the attraction for teens with Jott seems to lie in the way that P2P text messaging is integrated into the app.  That’s an activity that appears to take up a sizeable chunk of teen life – over 100 texts sent a day on average:

And that ability to easily message peers directly within a network is the key. While apps such as Facebook, Snapchat and Instagram rank at the top for social networks among teens, texting reigns supreme. According to Niche data, about 87 percent of teens text daily, compared to 61 percent of those who say they use Facebook, the next most popular choice.

It’s tough to know why texting is the preferred method. What is out there is mostly anecdotal. Perhaps texting is simply the easiest form of direct messaging to one’s friends? Whatever the reason. They do a lot of it. More than adults. Girls send, on average, about 3,952 text messages a month, and boys send closer to 2,815 text messages a month, according to the Pew study.

Jott

Open Source, Cloud and DevOps

  • Richard Stallman (rms) is coming to London to give a talk on July 4th.  It’s a good opportunity to highlight the irony that rms who many view as the father of the open source software (not that he’d approve of that term) that powers much of the Internet doesn’t himself do social media:

Please note: Stallman has never had a Facebook account. He has never posted on Twitter, but has a Twitter account for posting comments on other sites. (Learn more.)

  • InfoQ profile Capgemini’s Apollo open source microservice and big data platform which represents perhaps the best collection of leading edge technologies for building a software-defined data center today:
    • Packer for automating the build of the base images
    • Terraform for provisioning the infrastructure
    • Apache Mesos for cluster management, scheduling and resource isolation
    • Consul for service discovery via DNS
    • Docker for application container runtimes
    • Weave for networking of Docker containers
    • HAProxy for application container load balancing
  • This InfoQ post highlights the curious lack of a definitive definition for DevOps and profiles two industry perspectives from IBM and Forrester Research respectively.  The first is more a description of broad DevOps concerns than a specific definition:
    • DevOps exists to help the business win
    • The scope is broad, but centered on IT
    • The foundations are found in Agile and Lean
    • Culture is very important
    • Feedback is fuel for innovation
    • Automation helps
  • The second is based around the use of the CALMSS acronym which offers a handy mnemonic covering the same sort of territory:
    • Culture – Own the change to drive collaboration and communication
    • Automation – Take manual steps out of your value chain
    • Lean – Use lean principles to enable higher cycle frequency
    • Metrics – Measure everything and use data to refine cycles
    • Sharing – Share experiences, successful or not, to enable others to learn
    • Sourcing – Solid sourcing strategy to extend the ecosystem

Asia

  • In more evidence of a VC-led investment boom in Asia, entrepreneurial Indians finishing studying in the US may find their prospects are better back home as opposed to staying on rather like Kunal Bahl the founder of ecommerce startup Snapdeal.com now valued at $5billion.
  • However for those stuck in the old mode of Indian IT, things don’t look so rosy according to this LinkedIn post which really seems to have hit a chord:

Employees doing tasks that can be automated, the armies of middle managers who supervise them and all those with mediocre performance reviews and without hot skills are living on borrowed time.

  • The advice to anyone in such a position is to reinvent themselves:

What’s happening in the industry is ‘creative destruction”. New technologies are destroying old jobs but creating many new ones. There is an insatiable demand for developers of mobile and web applications. For data engineers and scientists. For cyber security expertise. So for anyone who is a quick learner, anyone with real expertise, there will be abundant opportunities.

  • Talking of reinvention, Penang in Malaysia previously famed for its curry, is now the “Silicon Island” of Asia.  Its success is built on the Internet of Things with the attendant consequences of spectacular growth:

andaman quayside penang

Security

Breaches are no longer a technology issue. They’re a core business issue. According to a recent survey of 200 corporate directors conducted by the New York Stock Exchange and the security company Veracode, more than two in five respondents said that CEOs should face the brunt of any breach-related backlash.

We live in a time of rapid technology change. As software eats the world, it leaves more and more aspects of the world vulnerable to problems in the software – and vulnerable to problems in how that software is used, deployed, and updated. … Before long, security will be eating the world. Companies whose software systems fall short on security will be driven out of business.

The risk that a company will be breached and the data put to uses not intended by the company is not only there, but it is significant. This is a technology problem our industry needs to solve. One thing is clear and it is that the biggest companies are the biggest targets and the largest technology companies are (I would assert) the most savvy and adept at these issues. But the problem is incredibly difficult in a world of nation-states leading some attacks.

Wearables and the Internet of Things

when a camera is plugged in to be charged, the service would upload the footage to a cloud service, eliminating the tiresome task of manually transferring the data from an SD card into editing software. From the cloud service, customers could edit the footage and share it. … there could be a feature where long videos are cut into highlight reels, which if implemented, would facilitate more shared footage and that’s what GoPro wants.

Team-Oralce-USA-Bremont-Americas-Cup-3

Smart Retail

  • Facebook have made a major move into the smart retail space with the announcement that they are going to offer businesses free Bluetooth beacons through their Place Tips system.   The use of beacons for microlocation services is something many retail brands have been playing with.  Facebook’s entry into this space could be a watershed provided they are able to allay user fears around data collection and privacy.  An open alternative might provide some interesting competition:

Once a business – such as a coffee shop or restaurant – sets up a beacon, it can detect when a Facebook user is within a set distance. … The beacon can then send “fun, useful and relevant” information into the user’s News Feed.

Facebook beacon

Timpson's phone repair services

Hardware

  • The popular 4tronix Pi2Go robot for hobbyists is now available on Amazon.  Do yourself a favour and buy the fully assembled version – it costs only £12 more than the kit version which requires a non-trivial amount of construction effort which isn’t necessarily all that much fun, unless you especially like soldering. The local Code Club has one of these connected up to ScratchGPIO for the children to play with.  It’s also possible to access robot functionality directly via Python calls that wrap GPIO.

PiToGo

  • Dyson’s CYCS desk lamp claims to last 37 years with a price to match – it retails at $649 for the entry level model:

Using Heat Pipe technology, the lamp uses a system similar to what’s found on satellites to keep eight LEDs cool and prolongs their life. The heat is pulled away from the bulbs and directed through an aluminum heat sink that spans the length of the lamp’s shaft. Each of those LEDs sits in a conical reflector to cut down on glare and added eye strain.

Big Data and Analytics

  • This TechCrunch post outlines the business case for tools and services which seek “world-changing potential” rather than pander to the 1%. The specific examples provided aren’t all equally world-changing but the overall point is a good one, namely that democratisation of access to data is key to creating potential:

Data is power. Those who can wield data effectively or collect data where previously it was unavailable can fight powerful monied adversaries and force social change.

  • This excellent InfoQ article outlines the prerequisites for any IoT OEM wanting to build a service on top of their hardware. Although there is huge potential in IoT, any organisation seeking to build meaningful differentiation on top of sensor hardware needs to first of all do their groundwork.  Only when they have established the infrastructure, methods, security measures and culture to work effectively at scale with data can they move on to exploit it:

At the core, all of these possibilities require organizations to have foundational capabilities around their internal data hygiene and analytic infrastructure: data curation, ownership and quality standards, consistent enterprise architecture, cleanly integrated systems, automated data onboarding processes, and mature analytic expertise. Without the basics in place and well managed, it will be very difficult to rapidly react to and evolve new analytic and data management functions and abilities.

Robots

Twenty-five teams of university and corporate roboticists competed for the prize, which was first proposed in 2012 by the Defense Advanced Research Projects Agency. The robots were graded on their ability to complete eight tasks, including driving a vehicle, opening a door, operating a portable drill, turning a valve and climbing stairs, all in the space of an hour.

  • In spite of growing hype around recent progress in Artificial Intelligence and robotics, the advances on show seemed to be relatively modest and focused on tasks that involved human and machine collaboration.  Several of the robots such as IHMC’s Running Man demonstrated endearingly human qualities including falling over and struggling to pick themselves up:

The challenge showed that robots are getting faster, more autonomous, and more adaptable. It also showed that robots still have significant shortcomings. They need stronger actuators, more powerful batteries, better touch sensors, more dexterous grippers, and, until they get better at perception and object recognition, humans there to help them.

  • The $2 million prize was eventually won by a South Korean team and their robot KAIST seen here turning a valve:

Artificial Intelligence and Machine Learning

The biggest companies in the world will all be technology companies, and commodities and financial services companies that aren’t technology based will find themselves losing market share rapidly over the next decade. The biggest banks in the world will likely be technology companies emerging out of developing economies serving the 2.5 Billion people that today are without a bank account, but most of whom will have a smartphone by the end of the decade. In 30 years the biggest companies will be based around personalized AIs, new energy and infrastructure, and most probably Asteroid mining for rare metals powering technology. Mobile and digital money will be significant players too.

  • The more proximate concern raised by the increasing power and capability of Artificial Intelligence is technological unemployment.  A contrarian voice on HBR takes issue with those of a dystopian perspective suggesting instead that we would all benefit from outsourcing some jobs to robots and providing this rule of thumb for where to draw the line:

any task that has an output or outcome which can be pre-stated or even guessed, should eventually be performed by a machine. Humans should eventually be left to more or less exclusively deal with open-ended endeavors that generate new organic value (as opposed to efficiency derived value).

  • This TechCrunch post in a similar vein emphasises the importance of human-machine co-existence in driving the next phase of progress for Artificial Intelligence:

We use words like “super-proprioception” to describe this feeling of man and machine combining with a fluid interface to enable new capacities that man alone or machine alone would not be able to accomplish. … This is the ideal future of machine learning, where data is extracted only to benefit those it is extracted from, where machines don’t replace humans, but rather rely on human intelligence to translate patterns and behaviors into information that can change the world for the better.

 

  • InfoQ technical presentation from a Facebook AI Research engineer on the current state of the art in Deep Learning highlights the use of RNNs and GPUs to speed up computation.

RNN

  • RNNs have been at the forefront of recent hype around advances in Deep Learning with some suggesting they may be the starting point for human-equivalent AGI.  Ray Kurzweil still seems convinced that that ‘technological singularity’ will happen around 2045.   However he’s also looping cloud computing into another prediction for 2030 which provides a more blended human-computer vision as opposed to the presumably 100% non-biological singularity AI:

In just 15 years, Kurzweil believes, the human brain will become a hybrid of biology and technology, and we will “put gateways to the cloud in our brains.” And as the cloud becomes more and more advanced and is able to store increasing amounts of information, so too will our brains. By the late 2030’s or early 2040’s, Kurzweil said, the majority of brain function, at least in terms of information processing and thought processes, will be non-biological.

ai

Software Engineering

With the release of Swift 2.0, Apple has promised that Swift will be open-sourced under a permissive OSI-approved open-source license, and will run on not only iOS and OSX but also Linux.

  • The future of Swift is important for anyone involved in mobile application development and the iOS ecosystem.  TechCrunch went as far suggest it could represent the logical endpoint of current enterprise app development.  After all according to Apple “98% of Fortune 500 companies have an iOS app” today. The enhancement and open sourcing of Swift provides even more reason for Enterprises to embrace it as their primary mobile development language:

Thanks in large part to the ease of learning Swift – as compared to Objective-C -, the open sourcing of core libraries, and the ability for developers to create contextual apps on the latest versions of iOS, the opportunity to create apps, which can utilize the proactive features presented during the keynote, to drive employee productivity increases enterprises will likely accelerate, and expand, their native iOS development plans in the year to come.

  • TechCrunch on managing developers.   Like the man in the taupe blazer, you’re basically there to unblock the developers and translate what they are doing in terms that executives and customers can understand.  And just because you’re in charge doesn’t mean you’re in control:

The great irony of management is that the higher up you go, the less actual control you have. When you are but a humble coder, you make the computer do exactly what you want; when you’re a manager, you only hope that people understand what you want, and then trust/pray that they do it both correctly and in a timely manner.

WorstNightmare

Digital Enterprise

  • The global media industry has been forced to adapt business models for the digital age over the course of the last 20 years. The likes of Google and Facebook have profited hugely but many traditional newspaper print publishers have gone to the wall.   HBR reviewed the landscape and suggested that it contains a lesson for traditional incumbents everywhere – they need to concentrate on innovating their business model and building new organisational equity around that pivot.  Or face wipeout:

Historical strengths, like content creation, might need to be radically reconfigured or shared across organizations only with a great deal of care. New strengths, such as data analytics, might need to be built or acquired.

  • One way of growing that equity is by studying and working with the startup community.   Another is to buy it in.  Either way, a vital key factor in any business model transformation is a strong, aligned underlying corporate culture.   Merely having a strategy isn’t enough.  Culture trumps Strategy and in the ideal case leads to the right individuals being empowered to deliver.  PwC’s Strategy& digital consultancy unit this week published the results of a survey on the role of culture in organisation change initiatives underlining that message:

Findings also suggest strong correlations between the success of change programs and whether culture was leveraged in the change process — pointing to the need for a more culture-oriented approach to change.

Today, however, the most important capabilities and resources often lie outside of an organization.  So executives need to leverage platforms in order to access ecosystems. Therefore, the ability to manage operations and the capacity to inspire employees is no longer enough. Today, we must learn how to shape networks around a shared purpose.

  • A good example of the sort of outcome from exploiting networks of open innovation is outlined in this excellent Twilio White Paper.  It explains why taking a software-defined approach to Enterprise problems is allowing “Software People” to leapfrog legacy and realise dramatic change.  Twilio itself is a key part of that dynamic and is a key constituent of disruptive business platforms like TalkDesk:

“Software People are also using the cloud to change the definition of what’s possible inside traditional enterprises. They are rolling out everything from new security services to new call centers that are more flexible and cost less than their on-premises equivalents. For example, Intuit prototyped a two-factor authentication solution for its payroll services in just an afternoon, before putting it in place to protect the accounts of more than one million small businesses. Instead of distributing hardware tokens, Intuit sends verification codes via voice or SMS to its customers’ phones. The messages cost just a penny a piece—far less than the cost of managing an equivalent on-premises solution and a tiny fraction of the millions of dollars a similar system would have cost to deploy.”

Work and Culture

The more of life you experience, the more that experience makes clear just how little of all that  was, all that is and all that will be, you will ever have the slightest chance of experiencing; you must eventually begin to realize that you never did know it all and you will be “partly green”, innocent uncomprehending until the day you die. If you are truly lucky you will be permitted to continue to pursue learning with the break of every day.

  • Tim Cook made an important comment on diversity in advance of WWDC suggesting it was “the future of Apple” not just because it was the right thing to do in terms of representation but on business grounds: “the most diverse group will produce the best product“.   There are hints that the next Apple keynote will be much more diverse.
  • Fascinating Nick Carr piece on the “causal link between higher connection speeds and higher abandonment rates” and the profound implications of any such correlation on human behaviour:

Digital technologies are training us to be more conscious of and more antagonistic toward delays of all sorts — and perhaps more intolerant of moments of time that pass without the arrival of new messages or other stimuli. Call it the patience deficit. Because our experience of time is so important to our experience of life, it strikes me that these kinds of technology- and media-induced changes in our perceptions can have particularly broad consequences. How long are you willing to wait for a new thing? How many empty seconds can you endure?

  • In the run-in to the recent UK General Election, the editor of Bloomberg Business accused the main UK political parties of “not doing enough to champion digital issues that matter” insisting that “internet technologies and the digital economy come across as an afterthought in their manifestos“.  It’s a topic of growing importance in the US too and seems certain to become an increasingly important concern in the UK as digital technologies become an ever more integral part of our society and economy:

From a business perspective, there has never been a better time to start or grow a digital business in the UK or Europe. But these issues will affect businesses just as much as individuals. More broadly, politics is out of step on the skills debate. A sector crying out for talent, concerns abound around kneejerk policies on immigration and the EU. Broadband is important for startups, but if you’re ARM in Cambridge supplying chip technology to every smartphone in the world, you’re more interested in knowing where the next computer science grad or R&D budget is coming from.

The silent digital debate: why politicians cant afford to overlook technology

Week 30

$
0
0

Apple

Apple’s Q3 earnings reveal a company powering ahead largely off the back of rejuvenated iPhone sales:

Precise numbers of Apple Watch sales weren’t provided but according to The Verge from examining the financials “it seems like a safe bet that Apple Watch revenue was over $1 billion” suggesting that they “already took 75 percent of the smartwatch market“.  Benedict Evans reckons this equates to around 2-3 million units sold:

[Apple Watch] launched during the quarter, but indicated that revenue was well over $1bn, which suggests unit sales of 2-3m, possible higher (Apple also explicitly contradicted an estimate that’s been going around claiming watch sales had declined since launch)

Apple themselves preferred to focus on the overwhelmingly positive customer sentiment for their newest product:

Bajarin Apple Watch 2

The company now has a scarcely conceivable $203 billion cash in hand and remains set to be dominant for years to come with key suppliers like LG prepared to invest billions to support their product range:

LG Display on Thursday announced plans to devote almost $1 billion towards building a fourth OLED factory, one that could potentially be instrumental in keeping up with demand for products like the Apple Watch.

Apple have performed particularly well in China over the last year where their products have a luxury cachet:

chinaApple

Imitation is the sincerest form of flattery and the scale of Apple’s influence in China is perhaps best reflected in the sheer extent of the fake iPhone trade:

Chinese police have cracked down a Beijing-based company which allegedly made more than 40,000 fake iPhones worth about 120 million yuan (19.6 million U.S. dollars) this year.

Devices and Manufacturers

As part of the strategic review, Qualcomm will implement an “aggressive” cost-cutting plan to reduce annual costs from its fiscal 2015 levels of $7.3 billion by approximately $1.1 billion. The cost initiatives will come from layoffs, streamlining and reducing the number of offices, the company said.

  • Engadget pass a cynical eye over Ubuntu Phone in a hard-hitting review suggesting that despite years in incubation it is still a long-way from a consumer grade proposition to compete with low-cost Android and lacking a directive hardware partner:

Despite years of development, Ubuntu Phone still feels like an early beta, and I think Canonical needs to think long and hard about the implementation of Scopes and bump native apps up the agenda. There’s nothing wrong with trying to be different, but there’s a reason Android/iOS are so popular. Ignoring the headway they’ve made in refining the mobile experience is, in my mind, setting yourself up for failure.

OLYMPUS DIGITAL CAMERA

  • WSJ is reporting that a German auto manufacturer consortium will buy Nokia HERE for €2.5billion.

Google and Android

9% of the visits to our interstitial page resulted in the ‘Get App’ button being pressed. (Note that some percentage of these users already have the app installed or may never follow through with the app store download.)

69% of the visits abandoned our page. These users neither went to the app store nor continued to our mobile website.

“three years after its inception, Streamus is dead. It’s been removed from the Chrome Web Store after Google revoked its API key. … Sean Anderson, a 25 year old developer based in California bet everything on his tiny, but incredibly useful Chrome extension.”

Apps and Services

The following year, due to popular demand, Heinrich, released “I Am Rich LE,” a more affordable version priced at $9.99; it’s still available for purchase today. This time around, the app even houses a few features  … To date, the app has 67 reviews, and holds an average 2.5 out of 5 stars rating, though most of the one-star ratings are due to the fact that the app actually contains something useful. “Can you please make this app so it does absolutely nothing like the first one?” implores one customer. “I didn’t buy this app to get my money’s worth.”

According to Silversurfers, the most popular type of Facebook posts for the over 50s are nostalgia, human interest, and handy tips.

users communicate with self-destructing videos recorded by placing their phones on their hearts. … Beme offers no chance to review or edit videos: they’re just sent straight to users’ feeds, which are themselves minimalist lists of recent uploads. Users click and hold on a video to watch it, and once it’s been seen it’s “gone forever.”

  • News and feed aggregator Flipboard now has 70 million MAU and just raised $50 million in funding to “expand our engineering and sales teams as well as increase investment in infrastructure to support the growing number of readers and curators“.
  • In this long, by his standards, Benedict Evans post he surveys how the boundary between apps and messaging is increasingly blurred.  It’s also the nursery in which the next stage of evolution of workplace productivity software is occurring.  The future propositions that will break the hegemony of the Office paradigm are not going to be better Office copies, they will involve a different approach altogether that is much better suited to support modern communication-centric workflows.  For Evans, Slack is clearly a contender:

PowerPoint gets killed by things that aren’t presentations at all. The business need is met, but the mechanism changes. … Slack is effectively a networked file manager, but instead of folders full of Photoshop, Word or Excel files you have links to Google Docs, SAP or Salesforce, all surrounded by the relevant context and team conversation.

“though it’s very valuable for a Facebook or Dropbox to shoot for “stars” and build constellations, what we have seen from companies like Evernote and the asian messenger apps (LINE, WeChat) is that a “basket of apps” approach that leverages a common core resource between other apps might actually be a more scalable strategy.”

In the end, it fetched over 9.5MB across 263 HTTP requests. That’s almost an order of magnitude more data & time than needed for the article itself. … What the hell is all this stuff?

  • The answer is JavaScript to support 47 different third party sites including many “advertising and spyware” sites in the author’s view some of which he enumerates before giving up:

It finally stopped loading

“now that the Internet is increasingly mobile and companies are more sophisticated about tracking users’ history and preferences, technology is less about “pulling,” through Google searches, and more about “pushing,” through smartphone notifications that are impossible to ignore because they cause our phones to light up and go ding.”

Asia and Africa

Those investments are beginning to bear fruit, with mobile revenue accounting for 50% of total revenue in the first quarter of 2015.

  • Vietnam is a classic case of a developing nation that has “leapfrogged” into mobile through a revolution in smartphone ownership.  The same dynamic is being played out across many other parts of Africa and Asia and seems likely to fuel huge changes in social aspiration and outlook in both regions with consequences that are not entirely predictable. This mic.com article outlines the tremendous potential for mobile technology startups in Africa which “will never be cord-cutters because there’s no cord to cut“:

Security

  • Smart cars may be safer in conventional driving terms but introduce significant new attack vectors that ought to raise concern in respect of the potential for very serious zero-day (ie. unknown) vulnerabilities particularly in respect of some of the faster and looser players scrambling into a new gold rush:

In the near future, the biggest danger on the roads might not be other drivers, but some guy miles away sitting behind a keyboard, hacking into your engine.

  • It’s a theme echoed in this TechCrunch polemic taking to task WashPo for a recent editorial in which it demanded that “Apple, Google etc compromise the security of their users’ communications by building in back doors for law enforcement.”  As the author points out technology can do a lot of things but it isn’t magic:

Engineering is all about tradeoffs. Security, or “golden key” back door: pick one. You can’t have both. That bird won’t fly. It is mythical nonsense.

  • It’s an observation that a claimed 37 million nervous subscribers to legitimised adultery site Ashley Madison no doubt learnt the hard way after the revelation of a mass heist of credentials from the site.  They (or at least the subset of them that are real people vs. bots/employee accounts) would probably like to magic away the risk of having the most private of details exposed online:

The still-unfolding leak could be quite damaging to some 37 million users of the hookup service, whose slogan is “Life is short. Have an affair.”

Cloud, DevOps and Open Source

  • James Pearce is Head of Open Source at Facebook.  He recently gave an interesting keynote as OSCON in which he outlined the company’s extensive use of OSS:

reddit

Data Visualisation

Artificial Intelligence and Humans

contextual, AI-based services will be where the biggest players in mobile — Google, Apple and others — make their biggest investments over the next few years. Users will start demanding more context-based services within the next two years. If done right, they could be a game changer for mobile devices, and will make life easier to boot.

pastry-image

Humans continues to compel us and keep us wanting more. Even detractors of sci-fi will be hooked onto the show pretty quickly. … On the whole Humans has been dodging the clichés in a genre that has been done over and over.

  • Even so, this tweet has a certain uncomfortable feel to it especially in the light of the consumerism exhibited by the real humans in Humans:

Science and Space

  • Bloomberg on the “superhumans” that have genetic mutations that make them “impervious to pain and broken bones” and the inevitable commercial race to exploit their genomes.

Kepler 452 b is estimated to be 1.6 times the size of our own world, and resides in a clement, life-friendly orbit around a star in the constellation of Cygnus some 1,400 light-years away that is eerily similar to our own sun.

  • The confirmation only seems to add to the mystery of the Great Silence of the Fermi Paradox.  Namely if earth-like planets and the chemicals of life abound, why is there no intra- or inter-galactic evidence of intelligent life more advanced than our civilisation?
  • 46 years on, an amazing image of Apollo 11 being moved to its launch pad:

Management

  • Dave Girouard is a former President of Google Enterprise Apps. In this FirstRound post he underlines why speed is a habit that can be built up like any muscle through practice.  For Girouard the key to speed is rapid decision making and execution which echoes the “speed + simplicity” mantra of LinkedIn CEO Reid Hoffman outlined here:

What are the building blocks of speed? When you think about it, all business activity really comes down to two simple things: Making decisions and executing on decisions. Your success depends on your ability to develop speed as a habit in both. … The process of making and remaking decisions wastes an insane amount of time at companies. The key takeaway: WHEN a decision is made is much more important than WHAT decision is made.

  • The wide variability in management ability and the lack of sanction those being managed have over those that manage them seems to be the primary driver for an interesting decision at SumAll to submit all executive and leadership posts to democratic mandate.  Given the trend for assertion of democratic rights more widely right across society, it seems surprising that more companies haven’t adopted this approach.  Then again, turkeys don’t vote for Christmas.

When people have crappy managers and deceitful executives, they start to hate life, and things get evil quickly. … So, we came up with a solution: Let employees elect their own leaders and make all managers accountable to the people they manage; if leaders can’t earn respect and loyalty, they shouldn’t be leading a team.

Feeling Overwhelmed? You Are Far From Alone

Startups, Freelancing and Brand You

  • However, before older readers begin to get ahead of themselves, it’s worth remembering that experience can also get in the way of effective bootstrapping.  It’s an example of the need to be wary of the growing influence of the Einstellung effect (“teaching grandma how to suck eggs”) as you age.

With its fast-paced adoption rate, WordPress is the market in which you should move your entrepreneurial steps because an increasing number of users and businesses are looking for WordPress services and products.

  • This spiky and sparkling post about how to brand yourself starts by asserting what not to do.  Your LinkedIn profile should not drone on about your skills (“a made-up dogma that takes your fascinating, vibrant self and reduces you down to dust and gravel“) but instead focus on the narrative by which you choose to define yourself. Your story should explain what you care about, what you’ve accomplished, how your brain works, how you handle problems and what you’re like as a person:

You have a unique story. No one living on this planet and no one who has ever lived can tell your story — it belongs completely to you. Your story would make a great movie. It has plot twists and awesome characters. You are the star, of course! A resume is a personal document. The number one thing your resume needs is context.

Culture

men are so terrified of being emasculated that they’d rather create a hostile environment for women than let them join the field of play.

  • The annual WOMAD festival took place this weekend not too far from Bristol and Bath in the UK, coincidentally the base of a vibrant startup scene competing with London. The balance between human and technology seemed to be a theme running throughout in the surprising surfeit of electronic music best exemplified by proceedings on the Bowers and Wilkins stage.  A series of lectures and demonstrations highlighted the history of the electronic synthesiser which date back to Léon Theremin’s experiences with radio noise in WWI.  American pioneer Robert Moog built upon that work with his mighty landmark creation the Moog System 55 (below) which was showcased in its full majesty complete with oscilloscope traces outlining different VCO outputs.

MoogSystem55

  • Later on Hannah Peel played out an ethereal mix of analog and digital sounds on the same stage including appropriately accompaniment by a programmable music box with a punch card hand cut to the tune of Tainted Love:

The piece is essentially a bunch of old cap guns with servos that pull their triggers. A Raspberry Pi with an Internet connection fetches data on US drone strikes from www.dronestre.am and fires off a cap every time someone is killed. At the same time, the story version of the data is printed out in thermal paper that cascades onto the floor.

Week 37

$
0
0

Adventures with Docker II: Linking Docker Containers

In last week’s blog I outlined how to create and destroy a virtual machine (VM) on the Digital Ocean cloud using Docker Machine running on Windows. This week I’ll outline how to create a stack of Docker containers on that VM in order to bring up a vanilla WordPress installation. Assuming your VM instance is up and accessible via Docker, your next step is to try a docker run on the official MySQL and WordPress images in turn thus:

C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mysql
C:\> docker run --name mywp --link mydb:mysql -p 8080:80 -d wordpress

It turned out that this didn’t work in my setup.  In order to find out why, you need to run a docker logs command to check the container logs:

C:\> docker logs mydb
... 
2015-09-10 22:34:13 62 [Note] InnoDB: Initializing buffer pool, size = 128.0M InnoDB: mmap(137363456 bytes) failed; errno 12 
2015-09-10 22:34:13 62 [ERROR] InnoDB: Cannot allocate memory for the buffer pool 
2015-09-10 22:34:13 62 [ERROR] Plugin 'InnoDB' init function returned error. 2015-09-10 22:34:13 62 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 
2015-09-10 22:34:13 62 [ERROR] Unknown/unsupported storage engine: InnoDB 
2015-09-10 22:34:13 62 [ERROR] Aborting

The log reveals that the MySQL container has run out of memory. From reading around, this seems to be a common problem.  The solution is to switch to using a low-memory database configuration. @charltones has a suitable forked Docker setup using MariaDB (MySQL fork) that can be used to build this image:

C:\> git clone https://github.com/charltones/docker-mariadb-lo
C:\> docker build --tag="mariadb-lo" docker-mariadb-lo
C:\> docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
mysql latest 6762f304c834 31 hours ago 283.5 MB
wordpress latest 2ea81e5584d9 3 days ago 512.3 MB
mariadb-lo latest f2485761e714 2 weeks ago 302.2 MB
mariadb latest 187668a0b2c7 3 days ago 302.2 MB

After building it, you can retry the earlier docker run recipe to create a low-memory database container and then check the IP of the VM:

C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mariadb-lo
C:\> docker run --name mywp --link mydb:mysql -p 8080:80 -d wordpress
C:\> docker-machine ip dodocker 
45.55.172.85

You should at this point be able to navigate to the default WordPress installation at URL http://45.55.172.85:8080 of course remembering to substitute with your corresponding IP address.

You can still experience memory issues on account of using Apache especially in a small Digital Ocean instance so it is worth switching from Apache to an alternative nginx-fpm based web stack to save even more memory.  The fpm here stands for “fastCGI process manager”.   There’s evidence to suggest a staggering figure of up to 40 million hits a day can be supported by this architecture at a total cost of as little as $10/month on Digital Ocean.

In order to switch, it is necessary to build the official wordpress-fpm Docker image and an nginx-fpm web server for it.  For the latter, I used another starting point fork provided by @charltones.   This snippet outlines how you build the corresponding Docker images:

C:\> git clone https://github.com/docker-library/wordpress.git
C:\> docker build --tag="wordpress-fpm" wordpress/fpm
C:\> git clone https://github.com/charltones/docker-nginx-fpm
C:\> docker build --tag="nginx-fpm" docker-nginx-fpm

Docker best practice is to have each container do one thing only, though that seems to be frequently ignored.  In this case we can build a three-layer stack holding the database, web and app tier by employing Docker linking and volume support.  In the following snippet, three linked containers are being run from the above images:

C:\> docker run --name mydb -e MYSQL_ROOT_PASSWORD=xxxx -e MYSQL_DATABASE=wp -d mariadb-lomem
C:\> docker run --name mywp-fpm --link mydb:mysql -d wordpress-fpm
C:\> docker run --name mynginx-fpm --link mywp-fpm:fpm -p 8080:80 --volumes-from mywp-fpm -d nginx-fpm

The first time this was tried, the wordpress-fpm container failed to start and supplied this error message: “docker cannot start container no such file or directory”.  Fixing that required changing CRLF line endings in a shell script copied over from Windows to Linux as per this StackOverflow post.  You need to remove the existing nginx-fpm container and image and rebuild it after switching to LF line endings as follows:

C:\> docker rm -f mynginx-fpm
C:\> docker rmi nginx-fpm
C:\> cd wordpress/fpm
C:\> scite dockerentrypoint.sh -> change CRLF to LF!!
C:\> cd ../.. -> back to root for wordpress 
C:\> docker build --tag="wordpress-fpm" wordpress/fpm
C:\> docker run --name mynginx-fpm --link mywp-fpm:fpm -p 8080:80 --volumes-from mywp-fpm -d nginx-fpm

Now, finally, you should be able to bring up the default WP installation by navigating to the same IP-dependent http://45.55.172.85:8080 URL as before in your browser.  Under the hood we now have a site built on three separate linked Docker containers in a single VM.

Entirely coincidentally, while writing this post, I was hit with an mandatory database update after the root WordPress installation for the blog auto-updated from version 4.3 to 4.3.1!   Luckily I had saved the page content text to a text file a moment before.   It served as another reminder, if any were needed, of the importance of having a backup-restore regime in place for your WordPress  workflow:

WPdbupdate

Taken together the posts this week and last have respectively walked readers through a full recipe for: a) creating a cloud VM and b) automating the construction of a stack of linked containers to hold a low-memory vanilla installation of WordPress within it.  The next step from here is to configure that stack to handle backup and restore requirements.

Apple

The Apple Watch Hermes is unveiled. New Apple TV

  • A few more discerning analysts were able to peer beyond the product buzz to examine the iPhone Upgrade Program, a new move that could prove to be far more significant than anything else they announced in the long run.  The program seems specifically designed to further demote the role of network operators within the Apple ecosystem and represents “really bad news” for the likes of AT&T and Verizon:

With this brilliant move, Apple also cleans the pipeline for customers who are intrigued by a new iPhone but don’t know what to do with their current one. You only have so many family members who are happy with a hand down. The returning iPhones are good for Apple too. They can be refurbished and sold to a different customer segment or recycled for parts and materials.

iPhone upgrade program

  • Benedict Evans also focussed on the Upgrade Program highlighting another potentially profound consequence a subscription model could create – an extended market for the returned refurbished iPhones:

Rather than making an actual cheaper iPhone model say in the $250-$350 range, Apple just carries on selling models from 1 and 2 years ago at lower prices.

  • For years pundits have regularly exhorted Apple to make cheaper iPhones.  With this ‘cheap iPhones by stealth’ strategy as Evans calls it, they just did it without actually having to make any phones.  When this particular penny drops with Android OEMs it will be the mobile equivalent of that infamous coffee cup moment in the Usual Suspects…

Devices and Manufacturers

  • Another would-be plucky smartphone contender spun out of the ashes of Nokia’s Meego efforts, Jolla, also seems to be struggling. This week COO and figurehead Marc Dillon stepped aside with The Register suggesting that maybe their best hope lies in being chosen as the “independent” smartphone technology platform of choice by the Russian authorities.
  • Nokia themselves meanwhile increasingly seem set to turn to Android as their smartphone platform if persistent rumours are to be believed.   Engadget showcased some leaked photos of the C1, alleged to be their first of any sort since the disastrous (for those involved) divestment of their phone division to Microsoft.

The Nokia C1

Marshall dished out shots of JD and put on punk rock performances. At one point, an executive on stage proclaimed the company was going to [verb beginning with F] the competition, in the [orifice beginning with A]. Whether that’s going to catch on like Tim Cook’s “We think you’re going to love it,” I’m not sure. I’ll admit, though, it made me instantly warm to Marshall.

Google and Android

Google Play is available in China, but reaches only 21 million of an estimated 800 million Chinese mobile users, Warman said. The primary app stores of Internet giants Qihoo, Tencent and Baidu account for two thirds of the market.  … These players are unlikely to give away that advantage.

AndroidVersions2

Apps, Services and Design

  • Byte seems to be the hot app of the moment.   Built by the creator of Vine, it further blurs the line between applications and the media they create.  It’s a little tricky to get your head around but worth sticking with if only to gain a broader understanding of where app interfaces may evolve next:

“THERE’S NO REASON A UI COULDN’T BE A PIECE OF MEDIA.”

  • However, changing user interface with every release of your app constitutes “messing with your users” rather than progress:

Security

  • In equal parts depressing and unsurprising, this post by lava lamp analyses the contents of “paste as a service” proposition Ghostbin and finds it full of dodge:

Services that offer anonymity are great, and they solve some very interesting problems. Even so, anonymity is a tool that is often sought after and used by people without good intentions. While the numbers I’ve crunched above may contain some amount of inaccuracy, the picture remains clear: a significant amount of the pastes shared on Ghostbin have illicit content.

Percent Categories

DevOps

  • A great presentation from Pete Cheslock on how the ability of an organisation to leverage the benefit of devops largely depends upon the culture and structure of the organisation.  Which helps explain why the Romans would be awesome at devops:

  • From the presentation a great quote from Deming should serve to remind those that lead where to look first for problems:

Deming

  • Apparently WinOps is a thing and having a London Meetup next week to talk about among other things “Chef, Puppet, Ansible, Docker etc.”  Wonder if they’ll also mention CRLF-LF gotchas.

WinOps

Digital Enterprise

  • Another great post from Asymco which seeks to understand why Enterprise IT has been so sidelined by the social media analytics and cloud (SMAC) revolution.  As usual he nails it:

The problem is that enterprises have a capital use and allocation model which is obsolete. This capital decision process assumes that capital goods are expensive, needing depreciation, and therefore should be regulated, governed and carefully chosen. The processes built for capital goods are extended to ephemera like devices, software and networking.

Asia

  • Excellent TechCrunch post on why South-East Asia, not the US or Europe, is now the crucible for disruptive mobile business models. It’s a development that has been fuelled by a combination of “no-tail” leapfrogging of Web1.0 paradigms, a mass jump to a smartphone first market and open embrace of eCommerce over ad-based revenue models.

“Chinese visitors tend to plan upcoming shopping trips two to three months in advance … and research online price differences between luxury goods bought in their home markets and abroad.   This kind of shopping arbitrage has led to the rise of third-party selling agents, known as “Daigou”, which is fuelling the practice of savvy shoppers placing orders abroad.”

  • Interesting WSJ profile of Wingtech, one of a group of Chinese ODMs that started out in the wild world of Shanzai phone manufacture but are becoming increasingly technically sophisticated, automated and discerning both in terms of the work they take on and the calibre of their customers.  Wingtech’s  customers include Xiaomi, Huawei, Lenovo and Micromax:

Like most contract manufacturers, Wingtech does a lot more than just manufacture devices. It works with clients to design products, providing lower costs due to its large scale than what its clients could achieve by doing it all in-house.

In the last few years, hardware startup activity in Shenzhen has increased dramatically. It is now comparable to the software innovation system in Silicon Valley.

Wearables and the Internet of Things

  • The Raspberry Pi now has an official 7 inch capacitive touch 800×480 display that can be powered through GPIO port.  It’s available courtesy of ModMyPi for just over £50.  Interestingly there’s also an unofficial 10 inch screen from Kano with integrated battery that might be more suitable for media display projects.

Another technique—and the most intriguing—is bulk analysis of EEG streams, without knowing what the people submitting it are doing or thinking. With enough data and powerful artificial intelligence analysis, the system starts to find chunks of brainwave activity that are common to all people. Essentially, it starts to pick out the individual words and phrases used in the language of the brain.

  • Would you let these guys hack your EEG streams?

Artificial Intelligence

If you are sitting at a desk, driving a taxi or carrying a hod, stop for a moment and ask: could a robot or machine do this job better? … The answer, unfortunately for you, is probably – yes.

Each of these products is an example of a defining trend of our networked age:  the outsourcing of common sense to gadgetry. A foundational level of human perception and competence is being mechanized through apps and online services. The more mediated our lives become, the more we rely on media to make sense of the world for us. …  There is a fixed amount of intelligence in the world, and every time we think we’re creating new intelligence, what we’re actually doing is just redistributing the intelligence that’s already there.

intelligence

Software Engineering and Management

  • This “programmer competency matrix” offers a way to assess where your dev capabilities sit.  I’m not sure whether doing this blog represents level 2 or level 3 competency within the provided taxonomy:

Competency

  • InfoQ interview the author of Go Kit, “an open source microservice toolkit that can be used to facilitate and standardise the creation of Go-based services within the modern enterprise application stack.”

The greatest engineers who have brought us this far in software grew up in an entirely different era of computing — packed inside rooms filled with terminals, carrying mental models of new algorithm-based programs. Newer generations of programmers are born into a more fragmented world.

MOOCs

  • Whatever became of MOOCs?   This sharply observed post in The Kernel suggests that they’ve fallen to earth somewhat since the lofty rhetoric of 2011 when they were trumpeted as representing a “revolution in education”.  In fact, business model realities have forced a re-evaluation of priorities and MOOCs have essentially morphed into an “Uber for education” providing top-up domain specific training those who already have degrees. It helps contextualise Udacity’s introduction of nanodegrees:

MOOCs were lambasted for having a high dropout rate; the average completion rate still hovers around 15 percent, a level that would be unacceptable for a traditional face-to-face college class. And when the demographics of “successful” MOOC students were scrutinized in one University of Pennsylvania study, it was discovered that 80 percent already had college degrees.

Startups

  • FirstRound on startup scaling.  A kind of exponential growth characteristic of funded outfits that is a curious bedfellow to the their frequently “small agile team” culture origins:

There’s a unique feeling of ambiguity, chaos and stress that comes with doubling or tripling your team every six months. If you don’t manage scaling proactively, you can end up in trouble.

  • Crowdsourcing has proved an incredibly successful way of funding project development with success rates in the 90% range.   However, according to this Medium post, the biggest challenge the model faces is ensuring sufficient meaningful supply given it is “so foreign to most organizations that few are actually willing to let the crowd model work for them.

The democratization of the tools of production, combined with the scale of society and increasing amounts of free time, affords more people the opportunity to contribute to projects, in ways large and small, than we could have ever imagined.

  • TechCrunch on the need for VC to innovate itself into a shape driven by “investors who prioritize sectors that make a real-world impact increase their bottom lines.”  That includes greater geographic, racial and economic diversity.  Until that happens, the inconvenient truth behind the Silicon Valley hype will remain in place:

Most technology startups remain relevant only to the best-off in society, leaving out billions of people and trillion-dollar markets.

Culture and Society

  • Stumbled upon a Quora post by Barack Obama explaining why the US-Iran nuclear deal is a good one.  Quite aside from the politics it still somehow remains a surprise to see the US President engaging on a social question and answer site.
  • Sleep seems to have been getting a lot of coverage lately.  This contrarian Quartz post suggests it is a myth that we’re not getting enough of it or as the author somewhat drily puts it:

there is little reason to lose sleep worrying about not sleeping enough.

  • Staying on the theme of future past, an odd TNW opinion piece which suggests your future self “hates you” because you procrastinate and do things without considering the impact on you in years to come:

When I see someone smoking a cigarette that’s obviously someone who hates his future self so much he doesn’t mind if he gets cancer.


Week 39

$
0
0

Self-referential data visualisation I: Vincent

In January I published a post on some analysis I’d done on all my 2014 weekly roundups.  As we’re now around 75% of the way through 2015 I thought I’d revisit my blog stats.  Last time round I used a Python script to scrape the blog posts and then create the raw stats.  Excel was used to generate the graphs from these stats. This time around I wanted to try out the data visualisation pipeline described by Kyran Dale in his illuminating talk at PyCon UK in which he suggested using the Python pandas data analysis library for data preparation and switching to JavaScript for visualisation, specifically D3.js:

I took a few shortcuts in order to get something working in limited time.  In particular avoiding MongoDB as an intermediate JSON data store and using a visualisation grammar called Vega that simplifies access to D3.  A pure Python library called Vincent (see what they did there?) allows you to generate Vega directly from a pandas dataframe.  I’ve released an updated version of the original blog.py script imaginatively called blog2.py that partially implements the workflow Kyran outlined with these limitations built-in.  Running the blog2.py script output yields the following terminal output:

$ python blog2.py
A] Retrieve JSON stats from cached stats
B] Load JSON into pandas dataframe and clean it up
 week htmlsize wordcount
0 1 66331 3036
1 2 77970 4019
2 3 62670 2570
3 4 67475 2832
...
36 37 71646 3537
37 38 70583 3440
38 39 76900 3709
Sum of htmlsize: 2787803
Sum of wordcount: 132451
Average wordcount: 3396
---- EXECUTION TIME: 0.920894 secs ----

Plus the all-important graphic itself:

blogStats

Which is created in a file called ‘blog.html’ using Vincent direct from a reduced scope pandas dataframe rdf thus:

lines = vincent.Line(rdf)
lines.scales[0].type = 'ordinal'
lines.axis_titles(x='Weeks', y='Count')
lines.legend(title='Title Here')
createVincentHtml('blog.html','N',lines.to_json())

The data shows a constant average total 3.4k words per post per week and ongoing strong correlation between word count and HTML page size.  The next time I return to this topic, I’ll look to fill the gaps and explicitly switch to D3.js in the hope I can tidy up the graph.

Devices and Manufacturers

  • Who killed Nokia?  They did it to themselves they did according to research conducted by two Professors of Strategy at top European business school INSEAD.  The write-up is a pretty brutal indictment of a dysfunctional organisational culture which is a topic that returns below in respect of the VW emissions scandal:

Nokia lost the smartphone battle because of divergent shared fears among the company’s middle and top managers led to company-wide inertia that left it powerless to respond to Apple’s game changing device.

In a recent paper, we dug deeper into why such fear was so prevalent. Based on the findings of an in-depth investigation and 76 interviews with top and middle managers, engineers and external experts, we find that this organisational fear was grounded in a culture of temperamental leaders and frightened middle managers, scared of telling the truth.

Apple is a company that builds greatness because they built a process to make their customers better. They do it without permission and they do it without offense. But when the buyer perceives the change they feel in debt to the object. Getting payments on that debt of gratitude is how Apple is rewarded.

  • Blackberry’s first Android effort is revealed as the curiously-named Priv smartphone.   Positioned as a device that “combines the best of BlackBerry security and productivity with the expansive mobile application ecosystem available on the Android platform” it’s clearly aimed at the growing cipherphone opportunity.

  • Vertu launched a new flagship, the Signature Touch that “runs on Android 5.1 and packs an octa-core Snapdragon 810 chipset with 4GB of RAM, along with 64GB of internal storage plus another 64GB on the included microSD card“.

  • 5 million Raspberry Pi’s have now been manufactured by Sony UK at their Pencoed factory in Wales.  It’s a great, albeit all too rare, success story for high-tech UK manufacturing:

Google and Android

Nexus 6P is the first all-metal-body Nexus phone. Built in collaboration with Huawei, this 5.7” phone is crafted from aeronautical-grade aluminum, with a USB Type-C port for fast charging, a powerful 64-bit processor, and a 12.3 MP camera sensor with massive 1.55µm pixels (hello, better photos!).”

The practice of bundling products and services together may violate antitrust laws if a company dominates the market for a product that customers need, and then forces them to buy a complementary product or service

Apps and Services

  • Quartz on the evidence that if you are a US smartphone user “you only really use three apps on your phone” and one of them is Facebook:

The big winner, unsurprisingly, is Facebook. It’s a top-three app for almost 80% of users, according to comScore, and number one for almost half of them. (With 126 million US unique visitors in June, it’s the most popular app in the US by a solid margin. It’s also on almost half of users’ smartphone homescreens, according to the report.) And as Facebook goes deeper into mobile video, its share of time spent seems likely to increase.

TimeInApps

  • This presentation from Facebook engineer Simon Whitaker is a picaresque and at times jaw-dropping ride through Facebook’s codebase and tools initiatives. Presumably they have their reasons for embracing NIH with gusto and the deck provides reasonable justifications for each case but, man, are they building a substantial legacy for themselves.    They’ll need to keep hiring software engineers to look after all this and more:

SiliCon Valley

“there is a trend in Silicon Valley to focus only on solving the problems of the rich, whereas entrepreneurs in the rest of the world tend to focus more on the problems of ordinary people – and maybe it is that second sort of innovation that will prove the better long term bet.”

It’s really about love. And sacrifice. And hard work. And putting in the daily things that it takes to achieve great things. And how in the daily routine of being yourself, committing to goals and just living life, you realize that goals were easier to obtain than you had imagined. It’s a love story of sorts.

Dieselgate

  • Good HBR take on the VW emissions omnishambles outlines three vital lessons the company failed to absorb and now has to act upon really quickly if it is to recover:
    1. Being clean and green has real, bottom-line value.
    2. Protecting the environment builds trust, and trust is precious capital.
    3. Trust comes from transparency, and transparency is the norm today.
  • The key takeaway:

“being clean and green is important, but only if it’s authentic. And remember that everyone is watching you.”

SEPT15_23_454561263

  • Travis Bradberry in this thoughtful LinkedIn post goes further suggesting that what lies at the heart of the issue is nothing less than the integrity of VW’s leadership, which has this week changed.  He outlines some key leadership antipatterns including micromanagement, toleration of poor performance, cult of personality and the tyranny of the urgent:

The tyranny of the urgent is what happens when leaders spend their days putting out small fires. They take care of what’s dancing around in front of their faces and lose focus of what’s truly important—their people. Your integrity as a leader hinges upon your ability to avoid distractions that prevent you from putting your people first.

This sustained and sophisticated software scam suggests truly pathological levels of managerial desperation and contempt: desperation around a failed promise of clean diesel technology and unsubtle contempt for unsuspecting regulators and customers alike. Precisely how gullible did these wizards of Wolfsburg think people would be?

If Volkswagen knew that every customer who buys a vehicle would have a right to read the source code of all the software in the vehicle, they would never even consider the cheat, because the certainty of getting caught would terrify them

Software on TV

Artificial Intelligence

We define AI-powered business applications as software in which the user experience is driven by AI. Today, that takes the form of voice (Apple Siri), text-based interfaces (x.ai’s Amy) or intelligent notifications (Google Now). Other examples of AI-driven UX like Google Search’s autofill differ in that they do not automate a step within a workflow.

ai-chart-final-option2

Cloud, Infrastructure and DevOps

  • Hashicorp are one of the leading players in the DevOps commercial scene.  Founded just 5 years ago, the company is already behind key tools like Vagrant and Packer.  Now they’ve announced a meta-orchestration tool called Otto that seems to be positioned as a competitor to Docker Compose.

http <span class="token punctuation">{</span> 
    js_set $hello_world " 
<span class="token keyword">        var</span> str <span class="token operator">=</span> <span class="token string">'Hello World!'</span><span class="token punctuation">;</span> 
<span class="token comment" spellcheck="true">        // JavaScript </span> 
        str<span class="token punctuation">;</span> 
    "<span class="token punctuation">;</span> 
    server <span class="token punctuation">{</span> 
<span class="token punctuation">        .</span><span class="token punctuation">.</span><span class="token punctuation">.</span> 
        location <span class="token operator">/</span><span class="token punctuation">{</span> 
<span class="token keyword">            return</span> <span class="token number">200</span> $hello_world<span class="token punctuation">;</span> 
<span class="token punctuation">        }</span> 
<span class="token punctuation">    }</span> 
<span class="token punctuation">}</span>

By giving developers a way to express relationships between data rather than fixating on the data itself, graph databases offer a powerful new way to tame the growing complexity of big data.

Digital Enterprise

The rise of the citizen developer will be a common theme over the next few years, and enterprises that embrace the potential of Shadow IT will be best positioned to innovate and thrive.

Any significantly sized business will generate a huge amount of data from across its various operations, and the task of finding actionable intelligence hidden inside it all is not to be underestimated.

The evolution of wearable technology points to devices and chips which will be embedded by 2025, rather than worn explicitly. If a single chip is embedded within the human body, or encoded within cells, then in theory every human would become a hub or a node, addressable and traceable within the larger IOT ecosystem and every owned device assigned to that Human IP address. Your ‘social graph’ will become more valuable to the Government than your Social Security or NIS number and it’ll supercede these eventually. … In other words, humans could well become the ultimate ‘thing‘ within The Internet Of Things.

Wearables and Insideables

The world's lightest and thinnest smartwatch.

HUAWEI Honor Band Zero IP68 Water Proof Pedometer Sleep Monitor SMS Reminder Call ID for Android / IOS Smartphone E-475268

DARPA responded to a request from Fusion that “brain-neural interfaces” have not yet been implanted in soldiers, though test devices have been implanted in the brains of volunteers already undergoing brain surgery. We’ve changed the headline to reflect that implantation of chips in soldiers’ brains has not happened yet.

Space and Science

The researchers believe that an Earth-sized planet with an atmosphere similar to Earth’s would indeed spin as it rotates around small orange dwarf and some red dwarf stars.

Recurring slope lineae

The Martian paints what I believe is a much truer picture of our nation’s space program, one that requires the collaborative efforts of all teams in order to successfully explore whatever it is that’s potentially “out there.”

The Martian

Startups, Work and Management

EEengineering

  • FirstRound profile piece on GiveForward highlights the central role that a unique and authentic culture plays within the organisation and how it colours their approach to fundraising, hiring and retention:

Weird job descriptions weed out the boring, stuffy people and bring us the weird, creative and fun folks we want to work with.

They build trust by taking the opportunity to demonstrate their ability as they do their daily work, by asking knowledgeable questions and offering insightful suggestions. .. They build a team by using problems and crises in the daily work to remind members of the team’s purpose and what it values most. … They build a network by taking opportunities afforded by routine activities to build and maintain relationships with colleagues outside their group.”

Internal emails and interviews with people who’ve worked on that system show that as CEO Travis Kalanick and operations executives pressed engineers to build new features for the business, it came at the expense of stability in the back-end systems that power them.

  • And also provides this rather interesting insight:

Culture and Society

One start toward reclaiming conversation is to reclaim solitude. Some of the most crucial conversations you will ever have will be with yourself. Slow down sufficiently to make this possible. And make a practice of doing one thing at a time. Think of unitasking as the next big thing. In every domain of life, it will increase performance and decrease stress.

  • A tremendous Quora post on historical counterfactuals (“what if” scenarios) asks “what is the smallest amount of believable change in historic facts that would have had the greatest impact on the world as we know it today“.  One respondent suggests the haunting possibility of Czar Nicholas’ son being born without haemophilia and all that entailed:

If you want to know how major world events can hinge on one event?  That’s pretty much it… a baby born (Alexei) born with Hemophilia with a pair of loving and worried parents…. whose love and concern would throw the world into a century of cataclysm and catastrophe.

  • Staying on the theme of Russia, a recently-published book called The Red Web outlines how the country has essentially adapted the intimidatory methods and tactics used the old Soviet era to the digital age co-opting ISPs to host surveillance hardware and support their deep packet inspection goals.  The end goal being to establish “digital sovereignty” over Russian citizens. One wonders what privacy activist Edward Snowden makes of it all:

  • It’s not usual for a tech blog to review a pop record but then again, Chvrches are not an ordinary band.  Their first album, The Mother We Share, was widely feted not least for its emotional content.  Their second (traditionally difficult) album, Every Open Eye, is also a triumph according to the Verge.  The Atlantic review the band’s progress and the influence of Depeche Mode on their music – quite literally in the case of Clearest Blue:

“In today’s musical climate of disposable pop with little to say versus deliberately obscure electro where all hints of a topline are buried beneath layers of effects and fear of seeming mainstream, Depeche Mode still stand alone, unafraid to foreground melody and imbue music with emotion in a way few other songwriters can.”

Week 40

$
0
0

Self-referential data visualisation II: ggplot and nvd3

Last week I sketched out how to build a simple data visualisation pipeline to extract and clean some statistics from my blog posts before displaying them in a graph.  I used a library called Vincent to generate the visual representation.  This week I’ll look at another couple of other visualisation options, namely ggplot and nvd3.

ggplot

ggplot is at attempt to accommodate its widely-used R equivalent in Python with the aim of providing those coming from R a straightforward migration path:

“a plotting system for Python based on R’s ggplot2 and the Grammar of Graphics.  It is built for making professional looking, plots quickly with minimal code.”

ggplot is relatively easy to use and integrates well with Python’s pandas data analysis library.  However it is limited in terms of wow factor and not particularly Pythonic given its R heritage.  Using the same pandas dataframe from last week, we can generate a multi-line graph .png output shown below with this code:

from ggplot import *
wks = df['week'].astype(int).tolist()
counts = range(0,100001,20000)
gg = ggplot(aes(x='week'),data=df) +\
  geom_line(aes(y='htmlsize', colour='red')) +\
  geom_line(aes(y='wordcount', colour='blue')) +\
  labs(title="ggplot",x="Weeks",y="Count") +\
  scale_x_discrete(breaks=wks,labels=wks) +\
  scale_y_discrete(breaks=counts,labels=counts) +\
  xlim(1,len(wks)) + \
  ylim(0,100000) + \
theme_seaborn(style='whitegrid')
ggsave(plot, 'blog_ggplot.png')

blog_ggplot

nvd3

nvd3.js is a library that builds on d3.js with a specific focus on interactive charting being:

“an attempt to build re-usable charts and chart components for d3.js without taking away the power that d3.js gives you.”

The library is maintained by a commercial outfit, Novus Partners, and comes with a helpful Python library nvd3 which can be pip installed then used as follows to generate the nice interactive JavaScript-based HTML visual representation shown below.  Click here or on the image to get a closer look at the graph and generated code.  Note how clicking on the key circles nicely reshapes the graph:

from nvd3 import lineChart
with open('blog_nvd3.html','w') as html:
    chart = lineChart(name="lineChart", use_interactive_guideline=True, margin_bottom=60, margin_top=60, margin_left=90, margin_right=90, x_is_date=False,     x_axis_format='d',y_axis_format='d', height=450, width=1000)
    weeks = df['week'].astype(int).tolist()
    hs = df['htmlsize'].astype(int).tolist()
    ws = df['wordcount'].astype(int).tolist()
    chart.add_serie(y=hs, x=weeks, name='htmlsize')
    chart.add_serie(y=ws, x=weeks, name='wordcount')
    chart.buildhtml()
    html.write(chart.htmlcontent)

blog_nvd3

This visual is a good starting point and almost there but it is missing a few key features – the y-axis doesn’t start from zero, labels aren’t named and x-axis tick-values don’t show all weeks.  From closer examination of nvd3, fixing those problems would require changes to the library:

  • Specifying the y-axis range requires setting a forceY element in the JavaScript which the nvd3 add_chart_extras function doesn’t seem to be able to do.
  • It isn’t possible to pass an axisLabel into the nvd3 linechart initialisation.
  • It also isn’t possible for tickValues to be set up at  initialisation of an nvd3 linechart.

In order to verify fixes for these problems I edited the previously edited nv.d3.js JavaScript with the corresponding elements modified in place then wrote a function to generate just that code. The key adjusted JavaScript is presented below along with its graphical output.   As with the previous version, click here or on the image below to get a closer look at how it all works:

nv.addGraph(function() {
    var chart = nv.models.lineChart();
    chart.useInteractiveGuideline(true);
    chart.margin({top: 60, right: 90, bottom: 60, left: 90});
    chart.xAxis
      .axisLabel("Weeks")
      .tickFormat(d3.format(',d'))
      .tickValues(weeks)
      .scale(weeks)
      .orient("bottom");
    chart.yAxis
      .axisLabel("Counts")
      .tickFormat(d3.format(',d'));
    chart.forceY(0);
    chart.showLegend(true);

    d3.select('#linechart svg')
      .datum(datum)
      .transition().duration(500)
      .attr('width', %d)
      .attr('height', %d)
      .call(chart);
    });

blog_nvd3raw

There was an inevitability at the outset that this exercise would end up in hacking hand-crafted JavaScript.  Prior experience with Python visualisation libraries yielded graphs that weren’t quite right and required a descent into lower abstraction layers to sort out.  nvd3 arguably produces the closest thing to output suitable for the sort of visualisation pipeline highlighted last week.  However, even there, as demonstrated above, there are pixels the library cannot reach.  The only way to control them, short of modding nvd3, is by dropping into the lowest abstraction layer, namely nv.d3.js itself. It’s another inadvertent example of Atwood’s Law in action:

any application that can be written in JavaScript, will eventually be written in JavaScript.

If you’re interested in trying this out for themselves, I have released a new source file called blog3.py.  This script contains my latest self-referential data visualisation pipeline which includes full working code for generating the vincent, ggplot, nvd3 and raw nv.d3.js representations covered in the last couple of weeks.

Manufacturers and Devices

The 19.5-centimeter (7.7-inch), 390 gram (0.9-pound) Robohon sports a 2-inch touchscreen, a quad-core CPU, and can connect to 3G, LTE, and wifi networks – but who cares! It’s a freakin’ robot that makes phone calls, shoots pictures and videos, and sends emails!

In a nutshell, it’s a stripped back device that lets you call, text, set a reminder or set an alarm. That really is about it.

Punkt01front

Google and Android

Come November 18th, Google will have to change the requirements that it places on its hardware partners, specifically the ones that the Russian regulator feels are unfairly restricting apps that aren’t created by Google.

Apps and Services

  • Peeple was widely covered in the news for all the wrong reasons this week. The Verge referred to it as “YeIp for People”, a questionable proposition that lets users rate people as if they were Amazon purchases.   The negative feedback generated by initial impression prompted its CEO to take to LinkedIn to defend the product as a “positivity app for positive people” without really explaining why some see it as a cynical growth hacking experiment that ‘non-peeple’ can’t opt out of:

there’s currently no way for users to opt out of Peeple. Anyone can sign up anyone else if they have their cell number, and although only positive reviews are shown on the profiles of people who haven’t signed up, members of the public can’t see their reviews unless they join. It’s also not clear whether negative reviews are judged to be so based only on the star rating or whether the actual content is also taken into account. If just the former, it means that users could give people extremely negative reviews but a good star rating, with the targets of these write-ups never knowing about them unless they signed up. Call it a growth hack.

  • Interesting analysis from a long-form digital magazine called The Atavist on why they’re ditching their native mobile apps and going full web:

Now, after nearly five years and 51 stories in The Atavist Magazine—plus tens of thousands of publishers and individuals making their own on the Atavist platform—we’re discontinuing our native mobile apps to place all of our focus on the web.

With iOS 9, it finally seems possible to realize some of the major trends identified last year, like “invisible apps” that recede into the operating system and notifications as a primary interface.

From left to right, that’s what Apple calls Today, Notifications Center, and Spotlight Search / Siri Suggestions.

Hardware

a portable FM transmitter the size of a shoebox that starts working as soon as it’s connected to a small antenna, a power source, and an audio signal. Pocket FM resembles a radio receiver more than a transmitter, and a single device can air radio programs over a radius of about six kilometers.

Pocket-courtesy-MiCT

UHA Phase 12, photo Greg Beron

Security

  • Yet another Snowden revelation on the formidable extent of control security services can exercise over individual phone users.  Their armoury apparently includes ‘silent exploits’ operating at a lower level than the likes of Stagefright which allow them to take control of smartphones remotely without the user being aware of what’s going on:

“a specially crafted message that’s texted to your number like any other text message but when it arrives at your phone it’s hidden from you. It doesn’t display.”

  • Cue a picture of the GCHQ panopticon:

GCHQ

Military Artificial Intelligence

  • Annie Jacobsen’s “The Pentagon’s Brain” is a Google talk that provides an inside take on DARPA’s military research projects and objectives.   She’s clearly a fan but taken in the round, the direction she outlines is rather alarming and right in line with the popular Hollywood trope that the US military will be the first organisation to create Artificial General Intelligence (AGI) and that it will be embedded in a war-machine:

“Our weapons are heading to autonomous robotics”

  • It does all rather make you wonder whether the agenda here isn’t being driven by Hollywood plotlines.  To wit, a US Army spokesman talking about Tony Stark iron man exoskeletons:

  • DARPA are also investing heavily in neurotechnology which today is admittedly more science-fiction than fact as described in this review of the current state of the art.  However, that was also once the case with atomic weapons and we all know how dramatically their arrival impacted the globe which should serve as a salutary warning:

It seems the possibility of weaponization might not lie in some distant future—and there is ample precedent for the rapid transition of technology from basic science to disruptive, global menace. After all, just 13 years elapsed between the discovery of the neutron and the atomic blasts in the skies over Hiroshima and Nagasaki.

Cloud and Digital Transformation

  • Digital Transformation is now a Board and C-level issue for many companies according to this study from PwC which also highlights an interesting divergence in expectation between the CEO and rest of the C-suite:

Digital transformation becoming a board-room issue, PwC study says

Among enterprise users, iOS accounts for nearly two-thirds of the market. More than any other factor, that embrace accounts for the realignment

“As Slack ramps up development and companies spend increasing amounts of time within the service, it’s going to subsume some of those integrations and launch them as first-party features so you never leave the tool. … With teams that are all-in for using Slack, there’s no emailing links or attachments of documents, no futile time wasting trying to invite everyone to a group call. In the future, all office software will be in one place”

Wearables and The Internet of Things

  • AWS and the Internet of Beer sounds goofy but it’s actually a really great extended walk-through a real-world E2E IoT architecture.  A Raspberry Pi with GrovePi+ board and sensors connected to a beer keg serves as a proxy for an IoT device.  The rest is a standard web scale AWS setup.

It is amazing how fast this space is evolving. Hardware really is becoming more like software. In just a little more than a year, it has become far easier to go from an IoT idea to something that looks and works great. … The use of the AWS cloud for heavy lifting combined with the decreasing size and cost of IoT devices is a powerful combination.

Software Development

  • Elvis has left the building.   But not before ensuring proper code hygiene:

gitCommit

Blogging and Tweeting

one service has quietly emerged as the go-to alternative to Facebook and other big social media sites: the email newsletter service TinyLetter.  … Your list of email addresses is yours in a way few things online ever are.  … Sending a newsletter is as simple as posting a status update to Facebook.  … Anyone with an email address which is pretty much everyone, can subscribe to your newsletter. Journalists and techies have flocked to TinyLetter in recent years, slowly building its reputation as the hippest place to publish online.

  • This analysis of the language employed by Twitter users and what they choose to tweet on suggests a correlation with their earnings and social status:

People who are perceived as less anxious and religiously unaffiliated seem to have higher earnings, while those who have higher income tweet more anger and fear while posting more objective content. Lower income users swear more, and emotions including sadness, surprise, and disgust are more frequently associated with them than higher income users. Additionally, people who earn more tweet more about politics and NGOs.

Work and Startups

  • Most employees have no idea whether they are fairly paid or not according to a study by PayScale.  Two thirds being paid at market rate think they’re underpaid.  Seems like that only those who genuinely are underpaid relative to market rate know it:

W150921_SMITH_WHATWE

  • Controversial but arresting take from an entrepreneur on “why I’m seed funded and you’re probably not” calls out these four areas of focus in this order.  All too often 4. is elevated above the others:
    • The Team
    • The Market
    • The Business Model
    • The product or service you are planning to build/offer
  • Carole Cadwalladr of the Guardian published a colourful ride through the Silicon Valley tech startup VC scene.   She asks the question whether the dotcom bubble is about to burst again without going so far as explicit prediction.  However a distinct tone of unsustainability is clear in the repeated references to disruption for disruption’s sake and talk of ‘decacorns’ and ‘unicorpses’:

How many unicorns can you fit through the eye of a needle? Anyway, unicorns are over. It’s all about decacorns now. Companies that are worth tens of billions of dollars. … Companies in Silicon Valley think differently from the rest of us. Their ambition is of a different order. Their rate of growth is like nothing we’ve ever seen before. And it’s in San Francisco, where technology meets the real world, that we’re starting to see the beginnings of what this clash of civilisations will look like.

  • FirstRound post from Mike Arauz, co-founder of a startup called August that adopted holacracy as a total transparent methodology for organisational self-management.  It’s famously an approach used by Zappos, a much cited canonical example of the “exponential organisation”.  They’ve since dialled it down a notch but remain committed to the general approach.  Arauz suggests the following five fundamental principles for running your company in a radically transparent way:
    • Nail a memorable common purpose.
    • Digitize your org to tap your network and blow up the hierarchy.
    • Distribute authority and embrace total self-management.
    • Never stop iterating.
    • Work in public.

Culture and Society

Week 41

$
0
0

Self-referential data visualisation III: bokeh, dimple and Qlik

Over the last couple of weeks I’ve outlined a few different approaches for using code to generate a multi-line visualisation of size statistics for my blog.   I covered using ggplot, vincent and nvd3 purely from Python and also how you can descend into JavaScript with nv.d3.js.  This week I’m going to risk inducing visualisation fatigue amongst faithful readers by trying out three more approaches.  First up a popular Python library called bokeh followed by a JavaScript library that builds on D3 called dimple.js.  Finally I will try out a widely-used shrink-wrapped commercial product, Qlik Sense.

bokeh

bokeh is a library that’s often cited as a useful resource for rapidly creating graphs:

Bokeh is a Python interactive visualization library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of novel graphics in the style of D3.js, but also deliver this capability with high-performance interactivity over very large or streaming datasets. Bokeh can help anyone who would like to quickly and easily create interactive plots, dashboards, and data applications.

The library exposes a large API set and although the documentation is comprehensive and the design pretty Pythonic, it proved to be somewhat difficult to work out exactly how to style the output let alone explore interactivity options.  The code below worked with the same pandas dataframe used before to yield the results shown below:

import bokeh
from bokeh.charts import Line, output_file, show, vplot, hplot
from bokeh.models import FixedTicker

bokeh.charts.defaults.width = 1000
bokeh.charts.defaults.height = 600
line0 = Line(df,
  x='week',
  y=['htmlsize', 'wordcount'],
  color=['blue','red'],
  title="malm.teqy.net blog stats",
  xlabel='Weeks',
  ylabel='Count',
  legend=True)
wks = df['week'].astype(int).tolist()
line0.make_grid(0,FixedTicker(ticks=wks))
line0.make_grid(1,FixedTicker(ticks=range(0,100000,10000)))
output_file("blog_bokeh.html", title="")
show(vplot(hplot(line0)))

bokehPlot

dimple.js

dimple.js is yet another charting library that builds on D3.js:

The aim of dimple is to open up the power and flexibility of d3 to analysts. It aims to give a gentle learning curve and minimal code to achieve something productive. It also exposes the d3 objects so you can pick them up and run to create some really cool stuff.

If you’re unfamiliar with the territory, there are a bewildering variety of JavaScript charting libraries that build on D3.js including C3, nvd3 and Dimple.js.  This helpful post helps explain some of the background reasons why; not that it makes it any easier. Dimple is arguably the best one I found.  The following JavaScript template code was used as a Python string to generate the graph below it.  I was able to produce two separate y-axes to allow the lines to be nicely overlaid which better illustrates the correlation between HTML size and word count in blog articles.  Either click here or on the graph to get a closer look at both the interactive visualisation and generated code:

wks = df['week'].astype(int).tolist()
    width = 1000
    height = 600
    htmlsizes = df['htmlsize'].astype(int).tolist()
    wordcounts = df['wordcount'].astype(int).tolist()
    data = []
    for i,wk in enumerate(wks):
        data.append({"week":wk,"htmlsize":htmlsizes[i],
                     "wordcount":wordcounts[i]})
    title = "malm.teqy.net html and word counts"
    s = """<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min.js">// <![CDATA[
        
// ]]></script><script src="http://dimplejs.org/dist/dimple.v2.1.6.min.js">// <![CDATA[
        
// ]]></script>

“”” % (width, height, json.dumps(data), title) return s

dimplePlot

Qlik Sense

Qlik Sense Desktop is a popular commercial chart creation utility.  It occupies the same niche as Tableau in offering users a way of working with and visualising data without resorting to coding.  In order to do that it provides a clear and simple to use interface that can work with a wide variety of data sources and render and style a range of output graphs as illustrated here:

QlikSense

In order to set up Qlik Sense to process the information in the pandas dataframe, it is necessary to convert it to csv format first. This is very simple to do with pandas and the resulting output when fed into Qlik Sense and styled with an average line added yields the graph shown below:

df.to_csv("blog_summary.csv")

qlik

If you’re interested in following up on any of this, I have released a new source file called blog4.py.  This script contains full working code for generating the vincent, ggplot, nvd3, raw nv.d3.js, bokeh and dimple.js representations covered over the last three weeks.  Phew.

Devices and Manufacturers

  • The barrier to entry into the smartphone market has been getting lower over the last few years with a broad and deep software and hardware supply chain in China allowing entrants with ambition and money to build the ultimate brand accessory. Still, it’s something of a “jump the shark” moment to hear that Pepsi have also joined the club:

PepsiCo Inc said on Monday it is working with a licensing partner to market a line of mobile phones and accessories in China in the next few months.

“ClearForce enables OEMs to differentiate smartphones by providing customers with new dimensions in user interfaces such as speed scrolling, zoom, gaming, and text or photo editing by applying variable force with a finger or stylus. Synaptics® has been working closely with leading global OEMs and LCMs to deliver this new dimension in touch with force-enabled smartphones expected to ship in early 2016.”

One customer commands a huge share of the industry. You either do business with that customer, or fall behind competitors who do business with them.

Microsoft, it seems, is becoming a bit more like the old Apple, even as Apple becomes more like the old Microsoft.

  • Recode profile a photography startup called Light which is blending the images from multiple smartphone camera sensors . They’ve announced their first camera product, the L16.

20151007 Light L16 front

  • The Granite cipherphone which was announced at MWC has now been officially launched and is available to buy online.   Although the hardware looks like a commodity Archos offering, the software seems significantly differentiated from stock Android majoring on encryption features delivered by the underlying Granite OS developed by Brazilian outfit, Sikur.  They’re banking on customers paying a premium price for secure software, targeting 150k unit sales in 2015 and double that in 2016 for Granite:

The phone, conceived by IT security specialists from Sikur, and made by Archos, can now be ordered from its dedicated website for the hefty sum of $849. … The hardware you get for that is not unlike the existing Archos 50 Diamond.

Google and Android

FUMconclusion

  • It’s a timely intervention with security updates top of mind for many working in the Android ecosystem following the disclosure of the Stagefright exploit.   Some activist customers are taking to social media and blogs demanding updates.  This open letter to Motorola management includes an interesting suggestion namely shipping their products with a guaranteed certainty around the level of post-launch updates based on Gold, Silver and Bronze support ratings.  At least that way, the discerning customer would know what to expect.
  • TechRadar suggests that the refusal of Stagefright to go away is a vivid expression of Google’s fundamental lack of control over the Android ecosystem.  The scare is proving to be a nightmare for both Google and Android OEMs many of whom will have struggled to patch the first set of vulnerabilities only to find that Zimperium uncovered a second set.

The vulnerability that the fix will eventually fix evolves so that the fix doesn’t fix it any more.

  • Monday Note on Google’s Accelerated Mobile Pages (AMP) initiative designed to speed up mobile web access by delivering only the core HTML content and stripping out “javascripts, iframes, embeds, large chunks of the CSS etc.“.  The technology has huge potential to transform the experience of accessing content on mobile devices:

Depending on network conditions and the speed of your smartphone’s processor, the loading time is reduced by up to 80%-90%. I’ve been testing amp-pages for a couple of months now, it works spectacularly well.

389_amp_guardian

 

Apps and Services

It means you can — for the first time — experience an event unfolding in both real time and after it’s finished as they’re captured for experiencing it as it unfolded at any time.

  • It’s US only for now but the potential for the service is not lost on sharper analysts.   Stratechery wrote a great analysis pointing out that what Twitter have effectively done with Moments is reinvent the newspaper around the concept of story fragments – or ‘micronews’ if you like – which positions it in the same curated news contender space as Apple and Facebook:

“if the Internet broke down newspapers to their component stories, Twitter breaks down stories to their component moments, and those moments are chronicled not only by normal people on the ground but by the best news-gatherers on the planet.”

Moments screenshot 1

  • Ethereum has been having a moment of its own over the last couple of weeks being hit with the ‘next generation disruptive technology’ stick.  Billed as a “social operating system“, it’s a complex proposition to explain.   Founder Stephan Tual describes it as follows:

Ethereum is a platform that makes it possible for any developer to write and distribute next-generation decentralized applications. Borrowing the concept of distributed consensus and cryptographic proof that makes cryptocurrencies such as Bitcoin so effective in trustless payments, Ethereum extends the use of these technologies to trustless agreements.

Cloud and Security

  • InfoQ on Amazon’s recently published “AWS Well Architected Framework Guide” which surveys the four key pillars of any AWS install namely Security, Reliability, Efficiency and Cost.
  • Amazon have also formally jumped onto the IoT bandwagon with the announcement of a new platform called AWS IoT aimed at an increasingly crowded space.   In their case it makes a lot of sense – many IoT initiatives today aren’t really possible.  AWS IoT is built on a combination of legacy tech and the Amazon cloud:

The IoT platform is one of several to use a decades-old lightweight messaging protocol called MQTT. That technology has been outstripped in some use cases, but works well with devices because it can handle intermittent connectivity particularly well,

  • Fascinating engineering blog post from the founder of customer analytics startup Segment.com which details how they rebuilt their architecture around Docker and other CI/CD (continuous integration/continuous delivery) tech.

Artificial Intelligence

20151007-code-mobile-2015-

“The real risk with AI isn’t malice but competence.  A superintelligent AI will be extremely good at accomplishing its goals, and if those goals aren’t aligned with ours, we’re in trouble.”

  • With so many remarkable advances in artificial intelligence announced almost every day, HBR asks what it takes to stay human in the Robot Age.  As befits HBR, their primary focus is the role of management in the transition.  This observation doesn’t fill one with confidence:

this is the first great wave of technological evolution that is being driven by managers in the grip of the ideology of shareholder primacy.

Cars

  • Autonomous cars are coming:

Mars

Digital Innovation

  • This important research post looking at the growing gap between the most productive firms may be explained by differential abilities to conduct and absorb digital transformation.  It also provides an excuse to reference William Gibson’s famous quote:

“the gap between the globally most productive firms and the rest has been increasing over time, especially in the services sector. Some firms clearly “get it” and others don’t, and the divide between the two groups is growing over time. … If the future is here just not evenly distributed then increasing the speed of diffusion holds the promise of a highly productive catch-up period in which the average advances to the frontier.”

productivity

  • The Digital Catapult is a UK government-sponsored centre created to “rapidly advance the UK’s best digital ideas” with a view that intervention will lead to “new products, services, jobs and value for the UK economy“.   A key part of its remit is an explicit focus on data:

“Our focus is on data and metadata – on the data value chain. This is how we describe the ways in which people and organisations create, collect, transport and analyse data.”

  • Here’s a link to their Creative Content Showcase showcasing some of the talent they are supporting including Andrew Fentem whose product FlickPixels (“wallpaper for the IoT”) was recently covered in the blog:

Product Management

  • InfoQ just published a presentation by Sherif Mansour, Principle Product Manager at Atlassian responsible for their enterprise collaboration tool Confluence.  It contains some interesting insights into why building simple products is hard and can’t be achieved through “a beating with the simple stick“.   Mansour outlines how Atlassian uses personas, customer interviews and journey maps to surface underlying requirements.   Ultimately it’s all about how good you are at covering these three bases:

  • This related post outlining “how to design better products in 5 days covers broadly similar territory suggesting the following five steps to drive agile sprint activity:
    • Understand (clarify the problem you’re solving). 
    • Diverge (brainstorm what’s possible).
    • Converge (rank solutions, pick one).
    • Prototype (create a minimum viable concept).
    • Test (observe what’s effective for users).

Blogging

What matters instead is influence, and one way to build it is by guiding audiences through the chaos of so much content. Today’s there’s no better way to do that – and demonstrate influence — than producing an email people will actually open. … It’s about the problem of discovery in an overwhelming age.

Culture and Society

at GUADEC 2009, of approximately 170 attendees, I believe I was one of only eight women. Of the software developers working on the entire GNOME project at the time, I was one of only three.

W150918_SCHROEDER_TALKTOME

Week 44

$
0
0

Self-referential data visualisation VI: Double Docker

Over the last five weeks I’ve developed a simple Python-based data visualisation pipeline for my blog stats culminating in last week’s basic Flask app.  This week, I will further evolve that app by reorganising the code and adding templates and styling. This improved version will then be hosted on the same Digital Ocean instance coexisting with, but isolated from, the first by means of a second Docker container.

Refactoring, templates and CSS

Our starting point last week was a single app.py using a utility file called blog6.py.  In this week’s refactored web2 app, app.py has been modified to use Flask templates.  In addition the old utility file has been split into two separate ones called mongolab.py and graphs.py which manage the mongolab.com cloud database handling and graph data creation respectively.  The corresponding Flask templates reside in a new ‘templates‘ subdirectory and a simple CSS file, style.css, lives in a ‘static‘ subdirectory.  Here’s an example of how the new app.py invokes a styled template which uses jinja2 markup in this case to create a Vincent graphic:

@app.route('/vincent/')
def vincent():
    title = 'malm.teqy.net 2015 blog statistics using vincent'
    jsondata = graphs.vincentVisualisationData(df,width,height,title)
    return render_template('vincent.html',\
      title=title, width=width, height=height, data=jsondata)



<!doctype html>
<title>{{ title }}</title>

<link rel=stylesheet type=text/css href="{{ url_for('static', filename='style.css') }}">

<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>
<script src="http://d3js.org/topojson.v1.min.js"></script>
<script src="http://d3js.org/d3.geo.projection.v0.min.js" charset="utf-8"></script>
<script src="http://trifacta.github.com/vega/vega.js"></script>

<div class=page>
    <h1> {{ title }} </h1>
    {% block body %}
        <div id="vis"></div>
    {% endblock %}
</div>

<script type="text/javascript">
    // parse a spec and create a visualization view
    function parse(spec) {
        vg.parse.spec(spec, function(chart) { chart({el:"#vis"}).update(); });
    }
    parse({{ data|safe }});
</script>

The end result is structurally similar to last week’s example but more modular and consistently styled:

visualisation

Double Docker

A second Docker container has been built and deployed onto same the Digital Ocean VM running last week’s container using the following commands:

$ docker build -t flaskapp2 .
$ docker run -d -p=5001:5001 --name web2 flaskapp2

And now we have two Docker containers running separate Flask apps on the VM, the older one on port 5000 and the new one built this week on port 5001 verified with this Docker command:

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f59f538c1e72 flaskapp2 "python /usr/src/flas" 6 seconds ago Up 4 seconds 0.0.0.0:5001->5001/tcp web2
676f987989ce flaskapp "python /usr/src/flas" 6 days ago Up 6 days 0.0.0.0:5000->5000/tcp web

You should also be able to browse the second container at http://labs.malm.co.uk:5001.  To pull a copy of all the code for building both the web and web2 containers, run the following command:

$ hg clone https://bitbucket.org/malminhas/blog

There are several other directions that could be explored from here. One useful one would be to replace the cloud mongolab database with a local redis store. However, I’m going to park the extended data visualisation with Docker exercise here for now.

Mobile Devices and Platforms

smartphone-market-pyramid-research

Apple’s increasing monopoly on the high-end of the market is creating a virtuous cycle that ensures they will own the high-end indefinitely. From an app perspective, new and updated apps launch first on iOS, which means people who care buy iPhones, which means future new and updated apps launch first on iOS. From a component perspective, Apple is increasingly the only manufacturer that can even afford to buy the best components, and they have massive scale which ensures they get first dibs on what is new.

And Apple’s iPhone plans are clearly working. The company announced that 30% of iPhone sales last quarter come from Android switchers. It’s likely that many iPhone buyers were able to take advantage of various iPhone upgrade programs available from retailers – Apple’s program only launched last quarter as well – and score contract-free new iPhones for cheaper prices than they’d be otherwise used to.

  • Meanwhile Android manufacturers remain under the cosh.  HTC “has stopped issuing future guidance going forward on its financials” citing “uncertainty”.   LG’s successes to date in 2015 were reversed in Q3 as “the firm’s mobile division lost $67.8 million in the last three months“.   And this leaked internal slide reveals what passes for innovation at Samsung and all too many other OEMs it seems – tweaking Android until it looks like iOS:

Samsung's internal slide deck comparing to the iPhone

  • Innerexile have a “magical” instant repair screen protector for the iPhone6s that uses a “hydro-oleophobic coating”:

So what’s the secret sauce behind this new coating? Well, it’s dotted with microcapsules that contain an adhesive-like liquid, and when damaged, the liquid will fill the void so quickly that you probably won’t even realize you’ve just scratched your case or screen protector.

innerexile-instant-self-repair-screen

Google and Android

“Combining the two operating systems means setting up Android to run on laptops and desktop computers, which would require big changes, as well as supporting the Google Play Store”

  • Meanwhile Chrome for Android has a new eight tab view.  Here it is on my phone together with a still image from the George Boole Google Doodle.

Chrome

microsoft-arrow-launcher

people don’t think so much about the operating system on their phone. But those who produce phones or sell phones or develop applications, they are very preoccupied with the operating system.

Mobile Apps and Services

  • Staying with Microsoft, they’ve also added polish to their mobile Outlook app suite offering built on the acquisition of Acompli last year and rebranding of its technology.  That allowed Microsoft to bypass all the tribulation of building Outlook from scratch for rival platforms.  By all accounts it’s been a spectacular success and vividly demonstrates why acquisition can often be a strategic game changer.  Microsoft are expanding on their foothold by integrating further acquisitions including the Sunrise smart calendar.  The screenshots below show Outlook on Android on the left and iOS on the right:

[Outlook] has almost 30 million users on smartphones and tablets, who use the app for 1.2 billion sessions a month, each 22 seconds long on average. Stats on mobile email market share are tough to come by, but that represents a huge jump from Acompli, which had fewer than 200,000 users before it morphed into Outlook. The company has continued with the strategy it started with the Acompli acquisition by buying the Sunrise smart calendar app and Wunderlist task manager.

“you can use the app to be notified when people change email address and other information, and update your records for them easily across all your connected address books”.

FullContact

  • TNW opinion piece from a TNW correspondent on why he dropped Dropbox “and you should too”.  Dropbox, rather like Flipboard, is in danger of finding itself dangerously placed at the nexus of the colliding tectonic plates of the giant platform players:

As Apple, Microsoft and Google have invested in integrating their cloud storage neatly into their ecosystems, it’s gradually negated the need to use Dropbox at all, and made its pricing far less appealing.

  • Bulk SMS gateway API vendor Nexmo will power KLM’s integration with WeChat.   Which reminds me, I looked at it exactly a year ago.  Out of curiosity I just re-ran my “hello world” SMS sender Python script and it still works.  Props are due.   To use this script you need to register with Nexmo and get an API_KEY and API_SECRET.  SOURCE and DEST are phone numbers.
from nexmomessage import NexmoMessage

msg = {
'reqtype': 'json',
'api_key': API_KEY,
'api_secret': API_SECRET,
'from': SOURCE,
'to': DEST,
'text': 'Hello, world!'
}

print("Sending message:\n%s" % msg)
sms = NexmoMessage(msg)
sms.set_text_info(msg['text'])
response = sms.send_request()
print(response)

{u'message-count': u'1', u'messages': [{u'status': u'0', u'network': u'23415', u'remaining-balance': u'1.93960000', u'to': u'xxxxxxxxxx', u'message-price': u'0.03140000', u'message-
id': u'020000008665F94A'}]}
  • This exhausting but highly insightful FirstRound post distills the collective wisdom of Kamo Asatryan on how to grow your mobile app user base.  He is billed as “one of a small handful of people who have observed hundreds of mobile apps, thought deeply and scientifically about their mechanics, and determined what they could change to grow faster“:

In this exclusive interview, Asatryan not only provides the formulas he relies on to help mobile apps optimize their entire user funnel, but the counterintuitive lessons and mistakes he’s learned on the job. Anyone who works on a mobile app in any capacity has something to gain from this clear-cut look at what drives true, meaningful growth.

  • Astrayan’s eight key takeaways:
    1. Don’t over-educate your users.
    2. Bust unexpected sources of friction.
    3. Find opportunities for positive reinforcement.
    4. Find and focus on your best referral channels.
    5. Make referrals a valuable win-win.
    6. Consider double-sided incentives.
    7. Don’t rely on standard social media sharing.
    8. Aim for steady growth.

Artificial Intelligence

The ability to continue publishing and otherwise maintain a presence in the scientific community is the most important factor for top students making career decisions,

  • BrainPickings published a review of a book rambling through the varied views of leading thinkers on Artificial Intelligence and what it means to be human.   It’s a bit of a mixed bag with a fair degree of that most human of traits, contradiction.  This is from  a much longer quote from Kevin Kelly in which he elaborates on the reality that “the kind of thinking done by today’s emerging AIs is not like human thinking” and what that means:

“AI could just as well stand for Alien Intelligence. We cannot be certain that we’ll contact extraterrestrial beings from one of the billion Earthlike planets in the sky in the next 200 years, but we can be almost 100 percent certain that we’ll have manufactured an alien intelligence by then. When we face those synthetic aliens, we’ll encounter the same benefits and challenges we expect from contact with ET. They’ll force us to reevaluate our roles, our beliefs, our goals, our identity.”

Yes, humans can retrain over time, to learn new skills, in readiness for new occupations when their former employment has been displaced by automation.  However, the speed of improvement of the capabilities of automation will increasingly exceed that of humans.  Coupled with the general purpose nature of these capabilities, it means that, conceivably, from some time around 2040, very few humans will be able to find paid work.

Tech unemployment curves

The Internet of Things

Wearable developers are more likely than other IoT developers to be young (80% are under 35), female (9.7% vs 6.7%) and based in Asia (44%, almost as much as North America and Europe combined).

W151020_WILSON_WHATPEOPLE

  • The security holes in IoT need to be dealt with if we are to avoid catastrophic consequences given the scale increase in attack surface represented by having the “internet on things“.

Security

Getting confused between “sequential attack” and a SQL injection attack is an easy enough mistake to make for a firm outside the technology sector. But TalkTalk is a telco so customers are entitled to expect it to have a clearer grasp of standard infosec practices.

  • The FT also weighed in suggesting TalkTalk only had themselves to blame following what appeared to be a lethal combination of slack security and an alarmingly loose disclosure without any apparent lockdown:

“Interest in TalkTalk as a possible target for hacking may have been piqued after one of the company’s customer service representatives tweeted information that indicated the company stored customer’s login credentials in an unencrypted format, sparking more than 2,400 responses on the social media site, according to Recorded Future. Discussions about TalkTalk’s unencrypted databases and at least 11 so-called cross-site scripting vulnerabilities took place on online forums used by hackers weeks before the actual attack on the company was announced.”

  • Perhaps TalkTalk should take note of this advice from APMG that responsibility for cyber security “should start and stop at Board level”:

Boards must  become fluent in the language of cyber security to improve  the way their companies deal with threats.  When it comes to cyber security, when does ignorance become negligence?

Software Engineering

First of all, not all 2B lines of code were in one language. Maybe about 50% of the code was in C++, 25% in Java, and 25% in python and other random small languages like sawzall and protocol buffer declarations. (Maybe Go as well, nowadays)

  • An awesome BSOD:

  • This TechCrunch perspective piece on why coding academies are “nonsense” makes for interesting reading.  There are some valid points in amongst the abrasive commentary.   The essential point being that coding day in, day out is not a career for everyone and is in any case exposes you to rapid skills obsolescence:

In 20+ years of professional coding, I’ve never seen someone go from novice to full-fledged programmer in a matter of weeks, yet that seems to be what coding academies are promising, alongside instant employment, a salary big enough to afford a Tesla and the ability to change lives.

Digital Enterprise

W150210_CHAKRAVORTI_COUNTRIESBUILDINGDIGITAL1

Science

Startups

  • You’ve heard of DevOps.  This FirstRound post extols the virtues of Sales Ops.   When done right they can make the difference between success and failure.

unicorn-exits-1024x583

Work

  • This must read post from Tim O’Reilly is part of his series on what he terms the WTF economy.  In it he outlines why high freedom environments “where employees are given a great deal of discretion to “do the right thing”  for customers, and where the company seeks to do the right thing for employees” represent the future of work.  It may be the only element of the psychological contract we can hold onto as meaningful in a work environment increasingly tilting to a transactional vs. loyalty-led model:

“The employer-employee relationship is broken, and managers face a seemingly impossible dilemma: the old model of guaranteed long-term employment no longer works in a business environment defined by continuous change, but neither does a system in which every employee acts like a free agent. The solution? Stop thinking of employees as either family or as free agents. Think of them instead as allies.”

  • The reality is that all too often companies are not very good at treating employees like people either as this HBR post points out. It underscores the importance of establishing a high freedom culture in the workplace:

There is little doubt that discretionary effort by people who are empowered to give their best produces not just better morale but also better products and services.

  • Even Google with its famed working environment has to deploy the Performance Improvement Plan (PIP) playbook on occasion.   In this dismissal detailed in a Quora response, the end came swiftly for a burnt-out engineer.  Least he got a coffee out of it:

In the end they didn’t wait for PIP to run through and gave me a termination notice in another week. I returned my Google hardware and badge, and my manager walked me out of the building – all in 10 minutes or so. My Google employee account was revoked at the same time. We went to a nearby cafe with him, and he even paid for a drink though :)

Culture and Society

If James Bond actually worked in MI6 today, he’d spend a large amount of time behind a desk doing paperwork and making sure everything was properly cleared and authorized.

  • Spectre involves a shadowy global criminal network that exploits information networks to build a digital panopticon.  Apparently even busy Bond villains manage full quorum face to face meetings.   The capitalist villains and cronies at the heart of John Kay’s fluent 45-minute Google Talk on “Other People’s Money” are arguably equally shadowy and omnipotent.  In his lecture, Kay covers the history and operation of the financial sector and the negative consequences of “financialisation” of the global economy over the last few decades. The title of his eponymous book comes from a 1914 classic that remains valid today:

The goose that lays golden eggs has been considered a most valuable possession. But even more profitable is the privilege of taking the golden eggs laid by somebody else’s goose. The investment bankers and their associates now enjoy that privilege. They control the people through the people’s own money

  • Quartz outline evidence suggesting that dancing with friends is good for your health.   Turns out Bez was onto something all along:

So the next time you find yourself at an awkward Christmas party or wedding dance floor, wondering whether or not to get up and groove, just do it.

Week 46

$
0
0

After Paris

The horrific events in Paris represent a watershed moment for European unity when reality hit home in the most terrible way that axiomatic belief in ever upward human progress should never be taken as given without struggle:

“The assumption of an automatically better future for the next generation is gone in much of the west.”

At this dangerous time, what we need now is more fellowship in others not necessarily more leadership:

The sentiment, that is, of sharing a common predicament even if we don’t share the same history, experience, or fate. A sentiment most necessary precisely when fragmentation and fundamentalism are far more common. Fellowship is an antidote to both, an alternative to otherness that does not imply sameness.

It’s a view shared by a French ex-hostage of IS, Nicholas Hénin who is clear on what they are after, namely create chaos and disorder:

They will be heartened by every sign of overreaction, of division, of fear, of racism, of xenophobia; they will be drawn to any examples of ugliness on social media.

Andrew Neil on This Week doesn’t think much of their chances and outlined what’s at stake in a blistering two minute Message:

One of the things that will inevitably be talked about in the wake of the Paris attacks is the role of technology in the rise of IS.  It seems clear that OTT Messaging apps with integrated 2P encryption will face new scrutiny with Telegram already implicated and scrambling to lock the stable door.  There are already calls in the UK to ‘fast-track’ the Investigatory Powers Bill as a result.  It probably won’t help the cause of objectors to have Edward Snowden onside:

The news that Anonymous are going to target IS also got a lot of coverage but there are suggestions that their involvement to date has had little material impact. Tragically in terms of unity, in many respects IS is clearly more effective at European integration than the EU.

Devices and Manufacturers

  • FastCompany on where Apple’s Head of Retail, Angela Ahrendts, is taking the company’s stores next after reportedly spending two years “listening and learning” to staff.  Two of the ideas she outlines have a distinctly Millennial touch – retail staff to learning coding and consume short video briefings:

One of the first things she decided to do was create a better way to communicate with her 60,000 Apple retail employees. “My kids were visiting from London and all they were doing in the car was WhatsApp and Snapchat,” she says. “It hit me: That’s the way we should communicate.” She now shares a video with her employees every week—three thoughts in under three minutes, she says—so her employees can feel more involved with Apple’s decision-making. … She also recently connected with Craig Federighi, Apple’s senior vice president of software engineering, for help coming up with a “curriculum to teach code in stores.”

ffox

490260_130909337682536163

Apps and Services

Facebook has basically taken the RSS feeds publishers used to have on their websites, renamed them, and made it so they can send push notifications to their followers’ phones instead of quietly updating in the background of those followers’ RSS readers.

  • Google’s About Me tool is handy for getting a dashboard view of the information has about you that you’re currently publicly sharing about yourself.

Security

stock-photo-quito-ecuador-august-white-smartphone-closeup-lying-on-laptop-keyboard-with-amazon-339998060

The way we daisy-chain accounts, with our email address doubling as a universal username, creates a single point of failure that can be exploited with devastating results. Thanks to an explosion of personal information being stored in the cloud, tricking customer service agents into resetting passwords has never been easier. All a hacker has to do is use personal information that’s publicly available on one service to gain entry into another.

  • Your unhashable fingerprints won’t help you – as Hackaday point out, they’re a really lousy password being both easy to copy and impossible to change:

If your work is using your fingerprint for authentication, your password is probably on your monitor right now.

Cloud and DevOps

  • AWS is a $7 billion juggernaut.  This LinkedIn Pulse article uses that to explore “seven secrets” of their cloud though the author isn’t able to find out very much from a rather stone-faced Andy Jassy.
  • Great rediscovered 2015 post from the ever-excellent Simon Wardley outlining why there’s nothing structurally that radical about devops really.  It started out as a novel practice moving to mainstream best practice and will eventually be superceded by whatever comes next.  We’ve been here before and will be here again:

  • For now, it remains a hot technology. Serving to underline that mindshare, news that Ian Murdock, the “ian” in Debian, is joining Docker.

New Relic’s quest to store all of our application logs is of a piece with, say, Google Photos’ push to be an unfillable digital shoebox, or Amazon’s urge to list everything that might be sold.

“I’ve been embroiled in a cordial email battle with Bank of America, literally for years, over my email address, which is simply null@nullmedia.com. Using null as a mailbox name simply does not work at B of A.”

GettyImages-134367495

Wearables and the Internet of Things

“We believe it isn’t about function. It is about function and design and branding,  We see tremendous growth opportunity and that a lot of our jewelry and watches could be connected devices”

  • This fascinating Al Jazeera film profiles Massimo Banzi, the Italian co-creator of Arduino in Shenzen.  In it he encounters the local Chinese Maker enthusiasts and providing some insights into why this tiny board represents a $100 marketplace and why Open Source is the key to its success:

Artificial Intelligence

Science

  • NewScientist published an article entitled “7 journeys of a lifetime” outlining several surprising ways in which you are a child of the cosmos.

Journeys of a lifetime: 7 stories of how the cosmos made you

Product Management

  • Must read post from HBR on what really makes customers buy a product.   It seems that peer observation is a significantly more important factor than “word of mouth” when it comes to consumer electronics and technology.

W151026_WILSON_WORDOFMOUTH

Startups, Unicorns and Innovation

There are growing signs that it’s getting tougher for private tech firms to raise money, particularly in sectors that interact with the physical world, such as food and grocery delivery companies.

  • Instead of writing pointless apps pandering to the “minor inconveniences of the cash-rich and time-poor” and wrapping it up with data science ninja narrative, this Guardian opinion piece suggests that “our best minds should be solving real problems“:

Many of these companies will fail, but here’s the real tragedy: the time and talent wasted on problems that aren’t really problems at all. There is something demented about such a huge flowering of creativity and innovation being focused on the minor inconveniences of the cash-rich and time-poor.

The battle between every startup and incumbent comes down to whether the startup gets distribution before the incumbent gets innovation.

Digital Learning

  • Future of Work episode showcasing another leading MOOC provider  Udemy:

The logic behind each exercise is tied to an actual game so that you get visual feedback and an actual reward when you solve an exercise.

  • Minecraft’s hour of code tutorial approaches the task from the opposite angle – a game becomes the medium to teach coding.

Digital Society and Culture

  • Facebook is now an integral part of everyday life for so many people around the world.  Nevertheless there has been something of a fashion recently to advertise and promote abstinence from its tentacles.  It’s all too much for one peeved  Guardian blogger who suggests anyone doing so shouldn’t feel ‘superior’ because they’re effectively missing out on modern life. Think I’ll continue to stay off it myself though not out of smugness, just time.

All media is social media and everyone is a critic. We live in constant TripAdvisor mode  …  I am a waste of space that you could not even be bothered to read, simply to slag off. Because there is no comment without comment. If a column is not retweeted does it even exist?

It’s important to remember that the introduction of Hello Barbie is just one part of a new interactive landscape in which nearly everything kids do is recorded and uploaded somewhere. Some parents have balked at such networked omnipresence, refusing to post any photos or otherwise identifying information of their kids online.

BOTSIssue_HelloBarbie_1000px-revised-1

  • 80’s music legend Prince was recently interviewed by the Guardian, if that’s the right term, in his native Minneapolis.  He comes across as an elusive and impish character, sharp as a tack, who doesn’t have much time for Apple Music et al:

Tell me a musician who’s got rich off digital sales. Apple’s doing pretty good though, right?”

the Black MIDI pieces play so many notes at once that we can’t really perceive them as such, and they end up sounding like a single synthetic timbre, and it ends up sounding even more like a chiptune than a piano. It’s a neat effect.

Week 49

$
0
0

In Search of a European Google

Recent evidence clearly points to Europe sliding into ‘digital recession’ relative to the US and Asia. Echoing the concern over falling behind, this Guardian post explores a variety of reasons why the chances of Europe creating a technology platform company to rival the giants of Silicon Valley like Facebook and Google are arguably slim to none:

The combined value of the top three internet companies in the Americas – so, basically, in America – is around $0.75tn (£0.5tn). In Asia, it’s around $0.5tn. In Africa, it’s $50bn. And in Europe, it’s just $25bn. … when you look at the top of the tech market, the very top, Europe is lagging a long way behind.

The reason the topic is important is because the large platform players like the GAFA quartet will increasingly become the gatekeepers of future tech power and scale employment prospects:

“This is as big a deal as the industrial revolution. You’re moving from the agrarian to the industrial to the digital age, and you’re seeing the digitisation of everything. Industries which weren’t considered to be digital are being transformed by some of the methodologies and the processes of the digital age. Take Tesla – we think of Tesla as being a ‘startup’, but it’s the same size as Audi.” Both companies are valued at around $30bn. …

If you’re engaged on any aspect of software technology in the UK within a wider global supply chain (which is pretty much everything), the visceral sense of being overtaken is something you are likely to have experienced at first hand in the work environment. It’s a dramatic volte face from as little as ten years ago when Nokia and Symbian were in their pomp and the US seemed a distant second in mobile technology terms.  Ten years on mobile now looks like a closed book – a digital Mount Rushmore with the 4 presidents replaced by Google, Amazon, Facebook and Apple.  Perhaps the Internet of Things will offer a chance for Europe to strike back.

Manufacturers and Devices

smartbatterycase

The biggest problem for the operating system was a lack of developer support. Faced with having to develop for a third (or fourth if Windows Mobile was on the cards too) operating system didn’t appeal to most time-strapped developers, and the range of apps out there was tiny in comparison to the competition.

Apps and Services

The app ecosystem has an extremely harsh power law where app adoption and monetization are heavily skewed towards the top few apps. It’s nowhere near 80/20. In fact, it appears to be more like 99% of the value is centralized to the top 0.01%. Let’s call it the app store 99/0.01 rule.

The amazing ways WeChat is used in China

  • And yet still they come, the would-be app contenders.  One startup seeking to help them get off the ground even more easily and disintermediate developers altogether is AI-app proposition Gigster who just received $10m in funding from Andreesen Horowitz.   They look well worth keeping an eye on:

The company finds top-notch freelance developers, designers, and project managers with pedigrees from MIT, CalTech, Google, and Stripe, and only accepts 5% of applicants. A sales engineer discusses proposals with clients, and using the AI engine, comes back with a price quote and production schedule in about 10 minutes. Then Gigster manages the entire development process through delivery of the fully-functional app.

built-on-gigster

For this “visual perceptive media” project, the BBC developed a mobile app that analyzes music listening habits and asks questions about users’ personalities. Paired with age and gender info, the resulting personal profile automatically tailors the short film to the perceived preferences of the viewer.

Digital Disruption and Cloud

  • It’s been covered before but the smartphone really is eating television and the trend is even more pronounced with millennials and tweens:

Nielsen media study 2

APIWorth

Big Data

Security

researchers say that they discovered that the app contained a number of security problems, including that digital certificates, which are supposed to confirm the legitimacy of the connection between the doll and the app, used a “hardcoded” password. Every app used the same password as part of this verification process

What do we mean by 'Penetration Test'?

  • Meanwhile this InfoQ presentation outlines what AI-based security attacks are and why they represent the “new frontline”.  Botnets are a particular focus:

greatseal-front

The Internet of Things and Wearables

  • The IoT TechExpo taking place in London Olympia over two days in Feb 2016 looks interesting if you’re into this area.

“an additional Blockchain component, for irrefutable trust and ID management, might provide equivalent DRM-like governance for IoT, and I see this as a natural evolution of DRM (or whatever you want to call it) for both ‘things’ and content.”

As IoT develops, SaaS will play an increasing role. Currently things like email, remote or virtual desktops, and CRM type tools occupy this category.  Think Salesforce* or Google* apps and you have the right idea for SaaS.  If you are looking for special functionality outside of your physical hardware, look to a SaaS type solution to fill this type of gap.

  • CNN ask whether smart guns are an answer to the mass shooting epidemic given the US doesn’t seem likely to reform gun law any time soon.

it just takes one look at the Carrera Connected to see how it is different than a Motorola or Samsung or Apple or what have you. It, well, looks like a watch. Specifically, it looks like a Tag Heuer watch, with its imposing size, angular edges, and bezel full of numeric markings.

Retailers and various other resellers have requested a total of 100,000 units of the watch, which is way more than what the company expected upon announcement

Tag Heuer Connected

Artificial Intelligence and Machine Learning

  • Google has launched its Cloud Vision API as a limited beta building upon the release of its TensorFlow Machine Learning platform.  It comes with a neat Raspberry Pi based technology demonstrator:

To show a simple example of the Vision API, we have built a fun Raspberry Pi based platform with just a few hundreds of lines of Python code, calling the Vision API. Our demo robot can roam and identify objects, including smiling faces. This is just one simple example of what can be done with Cloud Vision API

  • Long Atlantic piece on driverless cars and why they are shaping up to be the early 21st century equivalent of the 1960’s Space Race with Google, Apple, Uber and Tesla duking it out for supremacy in a multi-way high stakes shootout with Detroit-based car giants:

If fully self-driving cars reach the market, it’ll be in the world that smartphones made possible; a world of GPS-enabled maps, voice-controlled robot assistants, constant connectivity, and everything on-demand. And the leading players—Apple, Google, and Uber—are all vying for leadership in a new sector of this world, which they created. They know what it’s like to demolish the incumbent in an industry because they’ve all done it before.

  • First in a fascinating 4-part series of posts from Bob L. Sturm on the use of Recurrent Neural Network (RNN) based Deep Learning to “assist the process of music creation“.  His posts explore the results gained from a model trained on 23,000 Irish folk tunes.  Bob will be playing the music live tonight at Dorkbot in East London.

TheDoutlace

Innovation and Trends

  • As we approach year end, posts offering free predictions for 2016 inevitably appear on the horizon.  This HBR post predicts that “Algorithmic personality detection” and “bots” like Xiaolce will both be big.

Microsoft’s experimental Mandarin-language bot, Xiaolce, is akin to Samantha in Her. She lives inside a smartphone and has intimate conversations with her users, because the program is able to remember details from previous conversations. She also mines the Chinese internet for human conversations in order to synthesize chat sessions.

Companies are increasingly looking for new, dynamic ways to create and deliver corporate presentations, and PowerPoint just isn’t cutting it in the business world, says Pearson. Dynamic presentations with superior graphics and animation and the ability to be interactive is all the rage.

Software Engineering

  • IBM meanwhile have:

‘launched a Web-based Swift sandbox where you can write code and execute it in a Linux server environment. It lets you view your source code and results side-by-side, along with error messages in the output area.

You can now try Swift online without having to install anything

Work and Management

“new research suggests that if we’re not careful, making people aware of bias can backfire, leading them to discriminate more rather than less.”

Remember that creative problems are supposed to feel difficult. Most involve setbacks, failures, and that “stuck” feeling. It’s part of the process. Suppress your instinct to interpret these feelings as a signal that you just aren’t creative or that you’ve run out of good ideas. Reaching your creative potential often takes time, and persistence is critical for seeing a challenge through to the end.

Culture and Society

WalkerDistraction

One could say that the mechanism of metamorphosis is the only element of life that never changes. The journey of every individual, every country, every historical epoch—of the entire universe and all it contains—is nothing but a series of changes, at times subtle, at times deep, without which we would stand still. The moments of transition, in which something changes, constitute the backbone of all of us. Whether they are a salvation or a loss, they are moments that we tend to remember. They give a structure to our existence. Almost all the rest is oblivion.

Week 51

$
0
0

WordPress ‘SS-HeLL’

I was somewhat sidetracked this week in attempting to set up SSL support for the blog, something that’s been on the backlog for a while and is undeniably a good thing and clearly signalled as such last year by Google.  What I found was that adding basic SSL support to the setup was relatively straightforward.  Percolating it through properly to all the material published here over the last two years almost was a whole different matter and as yet incomplete. I’ve left both http and https access enabled for now while I try to work through the remaining https issues.  I thought it would be useful to outline the process I followed in case it helps anyone.

1. Setting up Apache

SSL support actually comes standard in the Ubuntu 14.04 Apache web server package being used to support the blog. You need to enable the corresponding module using a2enmod followed by an Apache restart to recognise the change as follows:

$ sudo a2enmod ssl
$ sudo service apache2 restart

At this point, you should be all set to follow the standard DigitalOcean recipe for creating an SSL certificate on Apache. How I did that is covered in the next section.

2. Installing an SSL certificate

Namecheap are a competitively priced SSL certificate provider that offer neat integrated chat support that I used during the process to resolve a couple of issues.  They offer approx £6/yr “PositiveSSL” certificate signed by Comodo which are ideal for securing a personal blog.  In order to get started you need to first create a private key and certificate signing request (CSR) using openssl as shown below. You get asked a bunch of questions to populate the CSR.  One of them is country name.  I found out the hard way that ‘UK’ creates an invalid CSR.  You need ‘GB’ instead:

$ sudo openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out malm.teqy.net.csr

Country Name (2 letter code) [AU]:GB

Once you have a valid CSR you submit it to Namecheap in a web form and then follow their Domain Control Validation (DCV) process.  It takes you about an hour to get a bundle of certs which you need to scp over to your target instance and then configure accordingly in the location you want the certs to reside:

$ scp malm_teqy_net.zip <user>@malm.teqy.net:/home/<user>

Now you need to configure Apache default-ssl.conf using this recipe. In particular you need to ensure these lines are added and uncommented within /etc/apache2/sites-available/default-ssl.conf and that you follow up by running the a2ensite command and restarting Apache:

SSLCertificateFile    "/etc/apache2/ssl/malm_teqy_net.crt"
SSLCertificateKeyFile  "/etc/apache2/ssl/malm.teqy.net.key"
SSLCACertificateFile "/etc/apache2/ssl/malm_teqy_net.ca-bundle"

$ sudo a2ensite default-ssl.conf
$ sudo service apache2 reload

 3. Getting WordPress working with SSL

Getting to this point was reasonably ok.  However, more work lay ahead to ensure WordPress (WP) behaved properly in SSL land:

    • Switch off WP Super Cache module to ensure that isn’t interfering with proceedings.
    • Modify WP /var/www/html/wp-config.php to ensure admin is always behind https which requires enabling this:
      define('FORCE_SSL_ADMIN', true);
    • Custom WP permalinks used in this blog don’t work out of the box with SSL. I used this recipe to get them to work modifying /etc/apache2/sites-available/default-ssl.conf and then restarting Apache to enable the fix:
      <Directory /var/www/html/>
           Options Indexes FollowSymLinks MultiViews
           AllowOverride All
           Order allow,deny
           allow from all
           Require all granted
      </Directory>
      
    • However a key usability problem remains.  Most of the pages are showing between two to five broken image links.  It turns out that my approach of linking to external images in my posts introduces problems in some cases where those links don’t exist under https. The solution to this lies in bringing them in from the cold into the blog media library.  I used the ‘image teleporter’ WP plugin to import images from external links to my local setup so they don’t appear broken in SSL context. However, that went disastrously wrong with the first attempt ending up filling my media library with nearly 2000 duplicate images before I stopped and uninstalled the module.  I did eventually manage to manually fix up about 5 or 6 of my most recent blog posts before running out of steam. They should now all work under SSL.

4. Next Steps

Right now it’s possible to navigate to the blog entry page either with or without https.  https does basically work and is now fully enabled for admin access.  However I haven’t switched off http yet because I still need to work out how to fix up potentially hundreds of broken external image links that aren’t available in the https version.  It seems likely that I’ll need to develop a script-based approach for recursing the content, identifying the problem links and importing the corresponding images rather than (shudder) doing it by hand.

This whole exercise has highlighted one of the fundamental issues involved in maintaining your own site. Namely that there’s often a lot of work involved that isn’t to do with publication per se but relates to system administration. Certificate installation, for instance, is probably second nature to many who work in devops land. However, unless you’re doing that sort of stuff everyday you’re going to be slowed down trying to get your head around script locations and the arcane DSLs (domain specific languages) involved.

Manufacturers and Devices

  • It’s a common refrain that Apple has historically operated as a hardware company that develops software to sell their products. They have now announced an Executive Team change that underscores the importance of hardware to the future of the company with the elevation of Johny Srouji:

“In nearly eight years at Apple as vice president of Hardware Technologies, Johny Srouji has built one of the world’s strongest and most innovative teams of silicon and technology engineers, overseeing breakthrough custom silicon and hardware technologies including batteries, application processors, storage controllers, sensors silicon, display silicon and other chipsets across Apple’s entire product line. Educated at Technion, Israel’s Institute of Technology, Johny joined Apple in 2008 to lead development of the A4, the first Apple-designed system on a chip.”

Apps and Services

Uber’s integration with Facebook is fairly straightforward. Over the next few days, Facebook will be updating its app so that existing Uber users can connect their accounts to Messenger and request rides from inside conversations in the app. New users can sign up for Uber from within Messenger, a potential boon to Uber’s growth prospects. (As an incentive to try ordering an Uber car through Messenger, the first ride up to $20 will be free for each customer.)

17messaging-web-articleLarge

Both Alipay and WeChat Payments benefit from social features and other perks that come from being tied into their already popular shopping and chat apps. For this reason, Apple Pay is unlikely to achieve the same scale as its Chinese competitors.

atlas_Nk8bQwhBl@2x

the majority of mobile applications launched in China struggle to retain users. In fact, most apps don’t engage users for more than a week,

202089

Artificial Intelligence

  • Boston Dynamics terrifying take on Santa Claus and his sleigh:

On one side, we’ve built a model of all the music we know about, that is powered by all the curatorial actions of people on Spotify adding to playlists. On the other side, we have our impression of what your music taste is. Every Monday morning, we take these two things, do a little magic filtering, and try to find things that other users have been playlisting around the music you’ve been jamming on, but that we think are either brand new to you or relatively new.”

Security

There is another consideration that might resonate at this time of year. Algorithms don’t need to take a holiday, so they can keep on working while some of their human masters are taking a well-deserved break!

Cloud and DevOps

For those that do not know, Red Hat is essentially an open source outside IT department for internal IT departments. They are the internal IT’s contact for support within Linux with the customer paying Red Hat an annual fee for that service.

_images/logo_redhat.png

dt151220

“Our mission, ultimately, is to have every employee inside of every company using Atlassian products every day,” says Atlassian President Jay Simons. “And when you consider that there’s more than 800 million knowledge workers around the world, that’s a pretty big ambition and it’ll take a while to get there.”

“I just thought you can’t outrun the internet. It makes no sense for us to spend the money to build and maintain an app where there’s probably applications out there that solve the problems,” she said.

IoT and wearables

“I have a Samsung RF4289HARS refrigerator.  The Google calendar app on it has been working perfectly since I purchased the refrigerator August 2012.  However, with the latest changes in Google Calendar API, I can no longer sign in to my calendar.  I receive a message stating ” Please check your email in Google Calendar website”.  I can sign in fine on my home PC and have no problem seeing the calendar on my phone.  Perhaps this is a Samsung issue, but I thought I would try here first.  Has anyone else experienced this problem and what was the solution?”

“Sounds like your fridge needs a software update to use the new API version.”

  • The Breitling Exospace B55 sounds like the sort of “notiwatch” you might find handy when flying your biplane to avoid having to fiddle around for your phone.  Breitling tout it as follows:

[a] “multifunction electronic chronograph [which] also receives notifications of the smartphone’s incoming emails, messages (SMS, WhatsApp) or phone calls (with caller’s name or number) as well as reminders of upcoming appointments.” This means the small LCD screen will display names and info when you get a message.

here’s what we learned in 2015: The most exciting wearables won’t look like computers. They won’t look like anything. Look down at whatever you’re wearing now. That’s the future of wearables.

The Falcon has Landed

It’s currently costing NASA $70m and change to deliver astronauts to the International Space Station and cargo can cost $10,000 a pound to get into orbit. The vast bulk of that cost is the rocket itself, which usually is destroyed in the launch.

  • Though of course it remains to be proved Falcon really is reusable:

The rocket will now be taken back to SpaceX headquarters and examined piece by piece. The hardware will be tested by x-rays and ultrasound to look for imperfections and may be fuelled up for a few static burns, if it is safe.

Software Engineering

  • Robin Wilson’s top five new Python modules for 2015 includes one which I’d not come across before but seems utterly indispensible particularly if you find yourself developing command line tools in Python.  It’s called tqdm and gives you progress bar wings.  Once you’ve pip installed, it, here’s an example I cooked up to show how it works:
from tqdm import tqdm 

def factorial(n):
   if n <= 2:
       return n
   else:
       return factorial(n-1) * factorial(n-2)
 
for item in tqdm(range(35)):
   factorial(item)

Work and Management

Top 5 Complaints of External Talent on Client Organizations
1. Organizations are too slow in making decisions
2. Organizations are too complex
3. Internal staff don’t work hard enough
4. It’s difficult to assess senior leaders
5. Sponsorship is insufficient, buy-in is weak and inconsistent

  • Contractors are an increasingly important fixture of the modern organisation and often the norm in certain high demand tech disciplines (eg. devops, data science, UX design).  Finding and managing this ‘Agile Talent’ is an increasingly important responsibility:

Accenture estimates that 20 to 30% of FTE’s are what we term agile talent (contractors, gigsters consultants, and other externals sought for their particular expertise); Deloitte estimates 30 to 40%. Our data suggests an even higher percentage in the future — over 50% of global companies surveyed plan to increase their use of agile talent.

  • Permanent staff meanwhile need to keep careful watch on how ‘collaborative’ their productive new silo-less organisation really is. The collaboration buzzword is very much en vogue of course but the reality, as HBR point out here, is often more more prosaic and typically centres around a few stressed super-connector producer individuals doing the heavy lifting for a larger group of passive consumers.

the distribution of collaborative work is often extremely lopsided. In most cases, 20% to 35% of value-added collaborations come from only 3% to 5% of employees.

On Skepticism

  • Rather than accede to criticism, he seems to have responded with vim:

Ultimately, the reputation economy is about making money. It urges us to conform to the blandness of corporate culture and makes us react defensively by varnishing our imperfect self so we can sell and be sold things. Who wants to share a ride or a house or a doctor with someone who doesn’t have a good online reputation? The reputation economy depends on everyone maintaining a reverentially conservative, imminently practical attitude: Keep your mouth shut and your skirt long, be modest and don’t have an opinion. The reputation economy is yet another example of the blanding of culture, and yet the enforcing of groupthink has only increased anxiety and paranoia, because the people who embrace the reputation economy are, of course, the most scared. What happens if they lose what has become their most valuable asset? The embrace of the reputation economy is an ominous reminder of how economically desperate people are and that the only tools they have to raise themselves up the economic ladder are their sparklingly upbeat reputations — which only adds to their ceaseless worry over their need to be liked.

We are now living in what I call The Age of Bullshit, or the Anthropobollockscene for short, wherein humanity’s powers of bullshit have irreversibly changed the planet for the worse. In the anthropobollockscene, men with pointless hats will sell you the sweatshop-produced goods you know and love for twice the price. In the anthropobollockscene, you will eat “Japanese tapas” at an “English gastropub,” and Instagram it. In the anthropobollockscene, you think you are happy.

GoatMilkChocolate

Myth 1: Screening saves lives for all types of cancer

Myth 2: Antioxidants are good and free radicals are bad

Myth 3: Humans have exceptionally large brains

Myth 4: Individuals learn best when taught in their preferred learning style

Myth 5: The human population is growing exponentially (and we’re doomed)

  • Richard Thaler is the co-author of Nudge and a Professor of Behavioural Science and Economics.  In this Google tech talk he outlines the flaws of the conventional ‘rational’ model used to predict economic outcomes.  Human beings are subject to bias, blunder and downright poor choice selection.  Accounting for that is vital to build a better model:

Culture and Society

  • Your identity is a construct and may be getting in the way of developing a true Weltanschauung:

These identity categories that we seem so fond of, which we believe make us more real, or more authentic human beings, are actually weighing us down. They limit us, make us immobile, and prevent us from moving forward as a united human race.

the country’s transition to something resembling democracy has come faster than anyone dared expect. For that, Myanmar wins the prize.


Week 4

$
0
0

Machine Learning

I recently completed the Stanford University Machine Learning online course available via Coursera and taught by Professor Andrew Ng. A companion post I’ve written surveys the course contents and provides some personal perspectives on the experience.  It includes a worked example of a data compression technique called Principal Component Analysis. The course exercises are conducted using the MATLAB environment whose origins go all the way back to Alan Turing, creator of the eponymous test for determining a machine’s ability to exhibit intelligent behaviour.

Machine Learning can be considered as a key subset within the broader field of Artificial Intelligence which this week marked a major milestone with the news that Deep Mind’s AlphaGo system had beaten Europe’s best Go player at the legendarily difficult game 5 times in a row.  Perhaps the most important aspect of this amazing feat lies in the cunning combination of AI and Machine Learning techniques being used to achieve it.  They are more sophisticated versions of the same concepts covered in online courses – notably neural nets and reinforcement learning:

After training on 30 million human moves, a DeepMind neural net could predict the next human move about 57 percent of the time—an impressive number (the previous record was 44 percent). Then Hassabis and team matched this neural net against slightly different versions of itself through what’s called reinforcement learning. Essentially, as the neural nets play each other, the system tracks which move brings the most reward

It’s an achievement with profound implications that have resonated around the tech world this week:

it’s conceivable that AlphaGo could play the game at a level that no human could ever attain. Hassabis said that games like Go represent perfect stepping stones for researchers to hit on the pathway to potentially creating something that could be considered a true artificial intelligence system. … Google has cracked what has long been held as one of the “grand challenges” in AI research, giving credence to the idea that true, general-purpose, artificial intelligence may be possible.

The Kernel’s Cyborg issue recently reminded readers that Roy Batty the leader of the runaway replicants in Bladerunner, was “incepted” in Philip K. Dick’s storyline on Friday January 8th 2016.

roy_batty

Go notwithstanding, we’re a long way from the fictional Nexus 6 but the date is an appropriate signifier that we are already living in a future where AI is big and going to get bigger.   Data from the US job market suggests demand is already beginning to eclipse other skillsets and that “AI skills are commanding the premium that big data know how used to“.

Artificial Intelligence on the loose

brain

“for those looking for a ride through our modern technological world, or indeed a preview of what is to come, this is it.”

  • Here’s something utterly inconceivable 35 years ago when the craze was at its peak.  The world’s fastest Rubik’s Cube robot built from commodity electronics components takes just over a second to solve a cube:

Facebook and its focus

  • Facebook’s sudden decision to wind down Parse its BaaS (Backend as a Service) platform must have come as a shock to the many developers who rely upon it.  Arguably it shouldn’t have done given it’s a canonical example of “platform risk” in action.  In what is something of a bitter irony, the suggested solution is for developers to host their own version of the Parse Server. Avoiding having to deal with devops was the whole point of them using Parse in the first place:

We are committed to maintaining the backend service during the sunset period, and are providing several tools to help migrate applications to other services.  First, we’re releasing a database migration tool that lets you migrate data from your Parse app to any MongoDB database. During this migration, the Parse API will continue to operate as usual based on your new database, so this can happen without downtime. Second, we’re releasing the open source Parse Server, which lets you run most of the Parse API from your own Node.js server. Once you have your data in your own database, Parse Server lets you keep your application running without major changes in the client-side code. 

  • The decision to close Parse is not so much about economics given Facebook’s recent glowing results.  It’s more about focus.  And Zuckerberg’s focus is on other areas.  A key one is Facebook’s hugely ambitious internet.org mission to connect the world.  Wired’s inside look at internet.org makes for fascinating reading.
  • Another key area of focus is Facebook Messenger which stands at the cusp of the next major evolution in user software:

Beyond customer service, it wants users to migrate to chat apps—and in particular, Messenger and WhatsApp, which it also owns—for every aspect of commerce, including discussing and tracking purchases and even buying things directly.

facebook-thumb

Researchers linked a high number of Facebook friends with feeling burdened or stressed out by the site.   our desire to lurk and unhealthily peer into the lives of those around us is built into the experience itself. …  the site [is like] an 18th century “panopticon”—a type of prison that allowed inmates to be viewed by guards at all times. “We know what all the prisoners are showing us from their jail cells,”

4930d929ab1f41816133b147b788e6bd

On average, folks had 150 followers but said that they could only count on 4.1 of them during an “emotional crisis,” and only 13.6 ever express sympathy

“as originally proposed by the social brain hypothesis, there is a cognitive constraint on the size of social networks that even the communication advantages of online media are unable to overcome. In practical terms, it may reflect the fact that real (as opposed to casual) relationships require at least occasional face-to-face interaction to maintain them.”

In Facebook’s case, the company has demonstrated its mastery of product focus and long-term commitment to user experience. … Meanwhile, a series of mediocre product changes at Twitter (such as the much-hyped but ultimately confusing Moments feature), a stagnant user base, and a massive executive brain drain have called into question whether Twitter can survive as a business.

Mobile Crossroads

The U.S. can’t compete with China on wages. It can’t compete on the size of the labor force. China has had a decades-long push in its education system to train these workers; the U.S. has not. And the U.S. doesn’t have the facilities or the proximity to the Asian component manufacturers.

  • In the world of Android, Samsung are being sued in the Netherlands by a consumer pressure group for a ‘lackadaisical approach‘ to Android updates. And in an interesting move, Google have announced a partnership with Movidius whose specialised Myriad “visual processing unit” may be able to support localised deep learning computation on the handset.

myriad2+(1)

“It won’t stop Microsoft producing a few handsets every year as a vanity project, but for everyone else it’s the end of the line.

The Internet of Things and Cloud

  • Good ContentLoop article outlining the key concepts behind microservices and includes this handy graphic of the journey to date from the first to the third platform:

151-3

Digital CIO mIndset slide.

Startups and Hiring

Thunderclap is a smart companion for any crowdfunding project. Think about it: your most loyal supporters are willing to donate money to fund your project. Those same supporters will likely want to donate their social reach to spread the word. That’s where Thunderclap comes in.

  • Every developer knows it always takes longer than you think even if you take that thought into account.  Jeff Atwood’s tweet echoes the wonder of Hofstaders Law:

Undercultures

4352

48411500.cached

On the shortness of life

The fact that death is inevitable is the most commonly mentioned hard to accept fact.

  • Paul Graham’s latest essay is a characteristically plain, powerful and essential read simply entitled “Life is Short“.  In it he outlines his coping strategies:

Relentlessly prune bullshit, don’t wait to do things that matter, and savor the time you have. That’s what you do when life is short.

The thought of death pursues me with a strange insistence. Every time I make a gesture, I calculate: how many times already? I compute: how many times more? and full of despair, I feel the turn of the year rushing toward me. And as I measure how the water is withdrawing around me, my thirst increases and I feel younger in proportion to the little time that remains to me to feel it.

Week 5

$
0
0

The Charge of the Chatbots

This Nautilus post on “why your next best friend might be a robot provides some useful insights into how rapidly AI-driven chatbots are evolving.  The article is written by a Microsoft VP working on the deployment of their Xiaoice digitial assistant in Asia.  It’s interesting how many users are already able to anthropomorphise their experience today with ever greater improvements over the horizon:

This could be the largest Turing test in history. One of its surprising conclusions is that people don’t necessarily care that they’re chatting with a machine. Many see Xiaoice as a partner and friend, and are willing to confide in her just as they do with their human friends. Xiaoice is teaching us what makes a relationship feel human, and hinting at a new goal for artificial intelligence: not just analyzing databases and driving cars, but making people happier. … We can now claim that Xiaoice has entered a self-learning and self- growing loop. She is only going to get better.

xiaoice

The acquisition this week of SwiftKey by Microsoft for $250million served as a further validation that a company with a clear focus and capability in applied AI and Machine Learning can have a worth far in excess of its size.  It’s possible that SwiftKey’s tech will find its way into a future version of Xiaoice. More generally her popularity underscores a growing belief among many analysts that chatbots are the next big thing in mobile and will eventually usurp the currently dominant mobile app paradigm:

building conversational bots costs less and takes less time than building and maintaining apps for iOS and Android platforms

Xiaoice is also an exemplar of the combinatorial change that is starting to reel in faster year by year.  MIT’s review of what robots and AI learnt in 2015 serves to remind one of some of the incredible progress made in several key areas over the last year:

Huge progress has been made in AI over the past few years, due to the development of very large and sophisticated “deep learning” neural networks that learn by feeding on large amounts of data, and this trend continued in 2015. The world’s biggest tech companies have hired experts in the field to apply the technique to tasks such as voice recognition.

Taken together with advances in other technical disciplines, ever more dramatic progress through combination seems inevitable:

@mtrends provided his perspective on the skills that will be needed to thrive in this world.  Critical thinking, complex problem solving and creativity are going to be key to navigating the shape of things to come:

The popularity of the Japanese virtual pop star Hatsune Miku is a signpost to the ground Chatbots might eventually evolve towards. What happens when they exhibit distinct personality traits and achieve fame on their own terms?

In the latest episode of The Brain, David Eagleman portrayed a vivid picture of neurons being physically sculpted by experience into brain circuitry that operates without any conscious control through out the course of your life.  The 100 billion neuron figure he uses is contested but irrespective of the actual amount, it does seem at least plausible that we will eventually be able to build something in software that approximates the physical structure. Given our limited understanding, it seems at best unclear what will emerge through experience from the underlying connectivity inherent in any similar substrate, be it software or hardware based.  Perhaps it will resemble a proto-AGI or go beyond into uncharted ASI territory. Advances like the MIT Eyeriss “AI chip modelled on the human brain” to allow localised AI use cases in IoT devices simply add to the witch’s brew.

Closely followed by the Babelfish

  • Text-based chatbots may arrive in force first but many of the tech enablers it employs are broadly applicable to voice interfaces once voice recognition has done its job.  All of which suggests the babelfish is coming. The Doppler Labs Here ear buds with ‘AR listening system’ looks a plausible harbinger boldly positioned by their parent company as a product that will “establish the ear as the focal point for the next phase of computing”.  Combine this with voice Natural Language Processing and the kind of multi-lingual Machine Learning driven translation offered by Skype or Google Translate and you can see how it might all come together:

Doppler wants every ear to be equipped with a mic and a computer, 24/7, so that people can hear exactly what they want, and only that.

And Virtual Reality

Many believe Magic Leap’s technology—along with a handful of competing virtual and augmented reality products—will usher in a sea change in how we use computers.

  • Magic Leap’s “chief futurist” is acclaimed sci-fi author Neal Stephenson. His classic 1992 cyberpunk novel ‘Snowcrash’ looked ahead to a world the likes of Magic Leap are actively working to create:

The Metaverse, a phrase coined by Stephenson as a successor to the Internet, constitutes Stephenson’s vision of how a virtual reality-based Internet might evolve in the near future.

  • However, it’s all about the content not the number of boxes sold:

Other signs of the future

Smart Mirror (Max Braun, Google)

Alphabet

  • A robot can now drive a golf ball faster than Tiger Woods in his prime and play a hole-in-one:

  • This tweet was widely shared this week.  It raises all sorts of thoughts about nature vs. technology not least how many animals that work with humans will increasingly face an uneasy and likely confusing coexistence with advanced robotics over the coming decades:

  • Many humans too are likely to find accommodation with advanced robots very challenging.  The Kernel’s baleful tale of the last days of HitchBOT who achieved celebrity for all the wrong reasons in 2015, offers a dark disturbed glimpse of a future foretold, the sound of distant thunder coming closer:

“HitchBOT was beaten and destroyed, stowed in car trunks, used in death to fuel viral content, and returned through the mail in a FedEx coffin.  … Without its head. The head, we have to assume, remains in the United States, where perhaps it is still smiling. Smiling on a shelf in the basement lair of someone—or something—who shall forever remain anonymous: this creature, moving stooped through shadows, breathing heavily, chewing on raw flesh.”

hitchbot_usa_3_hires

Apps and Services

  • WhatsApp – 42 billion messages a day with 57 engineers.

  • Last week’s roundup linked to a largely positive Wired article covering Facebook’s Internet.org initiative to “connect the world”.  This BuzzFeed post takes a more sober look at the struggle the company faces in India and how it’s Free Basics program was an approach that they stumbled upon through acquisition (Snaptu in particular) rather than conceived fully formed:

“Facebook’s massive push to bring the world online has hit a wall of activists and government regulators who argue that its free service violates basic principles of an open, free, and fair internet.”

Devices and Manufacturers

Infrastructure Costs

  • A Jenkins Cluster running on what look like 5 Raspberry Pis which would cost just over £100 at current prices:

  • At the other end of the spectrum but distinctly driven by the same cost imperative, the BBC has awarded a massive network infrastructure deal to BT kicking out Vodafone and Atos in the process.  The consolidation is the first step on what is likely to be an epic digital transformation journey:

Matthew Postgate, the BBC’s chief technology officer, said: “At a time when the BBC faces serious financial challenges, it will also save us tens of millions of pounds so we can focus more of our money on the programmes and services for licence fee payers.”

  • This sponsored article explains why China’s internet access is unreliable and what can be done about it.  The authors suggest that it is possible to improve performance with dynamic DNS trickery but the ultimate solution lies in investment by the Chinese authorities in stricter regulation:

“most people believe that all sources of all their problems with Internet in China comes from the existence of the infamous “great firewall of China” or from government intervention. Nothing could be more wrong. The source of most problems actually come from the way Internet is being managed in China by telecommunication companies and possibly from the lack of government regulation enforcement based on the principle of network neutrality.”

Surveillance Society, Truth and Encryption

  • Even without recourse to that sort of kit, it’s possible to track individuals in ways that many ordinary users probably don’t understand well.  The appropriately named cree.py is a free Python program that allows you to build a comprehensive portrait of a person’s movements over time by tracking their social media activity with all that entails:

The Creepy program visualises geolocation information fetched from Twitter, Instagram, Flickr and Google+, and allow one to get a complete picture of every social media message posted recently in a given area, or track the movement of a given individual across all these services.

  • Despite the recent fuss over end-to-end encrypted apps, there seems to be little incentive from either side to change the broader balance with respect to social media services.  According to the respected authors of this paper from the Berkman Law Institute at Harvard, social media service providers remain largely dependent on unimpeded direct access to user data for revenue streams.   It’s a key reason why Facebook and Google for instance remain able to read your messages.  On the other hand, for those apps that are encrypted, metadata remains accessible and provides a rich source of new surveillance data.  In fact, the coming explosion of IoT devices will offer them an entirely new channel to monitor individuals from afar with all the potential to drastically change the surveillance landscape for the worse.   The moral of the story?  End to end encryption has made life more difficult for security services but they still have ways to track individuals.  If privacy is of paramount concern, resist the urge to tell everyone what you’re doing on social media and watch what connected devices you bring into your smart home.

The Tech Product Ownership Battleground

  • Roman Pichler, the author of the excellent “Agile Product Management with Scrum” had reason to come up on my radar this week.  It’s worth reminding yourself of what a good tech product owner should be focussed on – presenting a clear vision of a product or service over multiple iterations is one of the key elements:

ProductOwnerRoleDescription

  • Steven Sinofsky outlines 5 ways in which small companies can compete with big incumbents.  Some great corporate jujitsu advice in here including the following.  It echoes a personal belief that scouring job adverts and GlassDoor reviews of competitors is one of the best ways of finding out what they’re really up to:

The incumbent’s org chart is almost always the strongest ally of a new competitor. The first step is to understand not only where your competitor is building out a response, but the other product groups that are studying your product and getting “worried”.

  • Even so, it’s a formidable proposition going up against an incumbent particularly a GAFA entity.  You’re fighting an increasingly impossible battle and there are arguably richer pickings further out from the white hot centre of the tech industry. In case you needed any further convincing, TNW published this sobering dashboard outlining just what you’re up against:

battle600

Work

  • HBR again on a new breed of employee driven by purpose and meaning and who “are willing to substitute a considerable part of the material wealth otherwise available to them to achieve purpose. Some actively try to reduce their worldly possessions; others have just realized that the big house and the extra car don’t make them happy. They would prefer making an impact“.

Software Engineering

there are two populations: one that finds programming a relatively painless and indeed enjoyable thing to learn and another that can’t learn no matter how good the teaching. The elephant in the room, the thing that Yvette Cooper, the ‘year of code’ or ‘hour of code’ people seem unwilling to admit is that programming is a very high aptitude task. It is not one that ‘anyone can learn’, and it is not easy, or rather it is easy, but only if you have the aptitude for it. The harsh fact is that most people will find it impossible to get to any significant standard.

  • The reality is that many developers even today are self-taught or learnt on the job and weren’t the product of rote training.  In that sense professional software development thus remains at heart a matter of skill and craft:

SO-dev-survey5

websitefun

Aliens

  • A SETI scientist explains why we haven’t found aliens yet and attempts to put a brave face on the Great Silence of the Fermi Paradox:

if you set the search space equal to the size of the Earth’s oceans, we’ve only examined one glass of water from those oceans for intelligent life. We’ve only looked at one small part of a very large picture.

  • Meanwhile it seems the search for aliens is getting weirder and more contentious with the arrival of METI – a rival organisation that wants to dispense with the precautionary principle and actively seek to message aliens.

Culture and Society

db_gu_books_fx_1000sq

Week 7

$
0
0

Days of Future Past

Almost exactly twenty years ago a UK-based computer manufacturer called Psion embarked in earnest on the creation of the Series 5, a landmark device that went on to spawn Symbian OS, the smartphone and all that followed into the twenty-tens.  I played a small part in that adventure spending much of the last few years of the 20th century working on foundational communications software development for a product that turned into an industry platform. Among other things that included the IrDA stack, a communication technology that was futuristic in 1996.  Twenty years on, it still retains a certain cult mystique for those that recall using it for dial-up access:

In today’s world, however, there’s a palpable sense of the approach of the post-smartphone era coming up fast and change in the air. The notion that we are living in interesting times in technological terms is reflected in this excellent post from leading tech commentator Chris Dixon.  In it he explains how the new shape of things to come is going to be driven by the duality of of small, cheap and ubiquitous hardware (the “peace dividend of the smartphone war“) and Artificial Intelligence. This combination of hardware and software will create a ‘Cambrian explosion’ of new devices and service propositions:

We’ll soon see significant upgrades to the intelligence of all sorts of products, including: voice assistants, search engines, chat bots, 3D scanners, language translators, automobiles, drones, medical imaging systems, and much more.

This seems a strong bet and reflects a significant upsurge in interest in cheap hardware hacking, AI and Machine Learning I’ve observed in the industry over the last couple of years.  Nonetheless, the view that smartphones will fade into the background as AI takes over may seem far-fetched to many.   Then again, dismissal of this sort of combinatorial advance has a long history – less than a decade ago Martin Cooper the ‘father’ of the cellphone was saying the same thing about the first iPhone:

Perhaps it’s not surprising given we humans are in evolutionary terms ill-equipped to absorb the true meaning of exponential growth and therefore of making accurate predictions that take it into account:

Projections

Given this backdrop, it is instructive to look at where predictions of the future go wrong and why.  Benedict Evans suggests as such in a recent post examining the smartphone era with the benefit of hindsight.  In it, Evans explains why the future is often inevitable in the long run but generally arrives via an unpredictable path often with a different set of passengers from what many expected:

It’s always fun to laugh at the people who said the future would never happen. But it’s more useful to look at the people who got it almost right, but not quite enough. That’s what happened in mobile. As we look now at new emerging industries, such as VR and AR or autonomous cars, we can see many of the same issues. The big picture 20 years out is actually the easy part, but the details are the difference between Nokia and DoCoMo ruling the world and the world as it actually happened. There’s going to be a bunch of stuff that’ll happen by 2025 that we’d find just as weird.

The future Evans is talking about lies in and around a trinity of double letter acronyms: VR, AR and AI, and it is as inevitable as previous tech shifts.  The real question is how it will arrive and who will benefit.  One clue from MWC where the tech is already getting more evenly distributed lies in this widely retweeted picture below. It speaks volumes of our times.  The blind masses experiencing bread and circus adventures in Hyperreality while the ‘real’ world passes by entirely undetected.  The ultimate triumph of inverted totalitarianism?

In inverted totalitarianism, every natural resource and every living being is commodified and exploited to collapse as the citizenry is lulled and manipulated into surrendering their liberties and their participation in government through excess consumerism and sensationalism

Artificial Intelligence applied

  • A week on from the Guardian’s effusive portrait of DeepMind founder Demis Hassabis, another Google “AI superhero” gets a profile.  This time it’s Android creator Andy Rubin and the tone of the article in Wired is more balanced and interesting.  What Rubin is seeking to do with his new company Playground is bring to life the world of sensors, AI and cloud that Chris Dixon describes above.  It’s nothing short of a total AI hardware platform proposition from the man who foresaw the dominance of an open mobile platform for smartphones and autonomous vehicles:

For AI to reach its true potential, Rubin argues, we need to bring it into the physical world. And the way to do that is to create thousands of devices that pull information from their environment: text and images, sure, but also sound, location, weather, and other sensory data. Rubin wants to fill the world with these data-gathering machines, the better to feed and train those massive neural networks.

The robot rat was created by implanting electrodes in its brain and mounting a wireless micro-stimulator on its back. These devices allowed a computer to communicate remotely with the rat, helping it assess the shortest path to take in a maze, avoid dead ends and navigate loops. Rather than remote-controlling the rat, the computer instead provided the rat with hints designed to help it make decisions.

  • Or indeed from eliminating all false positives in a confusion matrix.  Which may have resulted in many innocent deaths at the hands of US drones in Pakistan off the back of the NSA SKYNET (yes, they really called it that) program:

SKYNET engages in mass surveillance of Pakistan’s mobile phone network, and then uses a machine learning algorithm on the cellular network metadata of 55 million people to try and rate each person’s likelihood of being a terrorist. …. The 0.008 percent false positive rate would be remarkably low for traditional business applications. This kind of rate is acceptable where the consequences are displaying an ad to the wrong person, or charging someone a premium price by accident. However, even 0.008 percent of the Pakistani population still corresponds to 15,000 people potentially being misclassified as “terrorists” and targeted by the military—not to mention innocent bystanders or first responders who happen to get in the way.

Encryption and Privacy

eds

On the iPhone 5C, the passcode delay and device erasure are implemented in software and Apple can add support for peripheral devices that facilitate PIN code entry. In order to limit the risk of abuse, Apple can lock the customized version of iOS to only work on the specific recovered iPhone and perform all recovery on their own, without sharing the firmware image with the FBI.

  • Just because they can, however, doesn’t mean Apple should and that is at the core of the intense online debate with many tech oriented correspondents from Hackaday to Quartz exhorting Apple to resist.  With the likes of Donald Trump predictably ranged against their principled stance it’s hard to see any other position they can take when you consider the consequences of compliance:

The problem is one of consequences, of what comes next on this road paved with good intentions. If Apple gives in for just this one phone, there will be requests for more…and more. Of course, they’ll all be in the interest of solving the most abominable crimes, or for thwarting credible threats. How can anyone be against fighting terrorists, drug dealers, pedophiles…?

Blockchain

Blockchain

Apps and Services

  • Quartz launched an interesting and clever new ‘news’ app based on a bot UI model.  You ask it questions and it tells you the news.

Our users are moving away from Twitter in massive numbers. In the first month of 2016 we’ve seen Twitter usage drop by 2/3, while Facebook usage has remained constant.

But Stephen, these foul people are a minority! Indeed they are. But I would contend that just one turd in a reservoir is enough to persuade one not to drink from it. 99.9% of the water may be excrement free, but that doesn’t help. With Twitter, for me at least, the tipping point has been reached and the pollution of the service is now just too much.

Devices and Manufacturers

Cat_S60_Thermal_Camera_Smartphone

  • Talking of seeing in the dark, there is more going on with the announcement of $4 Android smartphone for India than first meets the eye.  Lauded as “the world’s cheapest phone”, the Freedom 251 is an Android 5.1 device with a frankly barely credible spec for that price.  It’s manufacturer is an unknown company called Ringing Bells.  IndiaTimes smells a rat – it could be a response to Facebook’s Free Basics or maybe even a political ruse but there’s no way even with today’s collapsing hardware prices you could sell this for $4 and make any kind of margin:

The Freedom 251 has a 4-inch display, 1.3GHz quad-core processor, 1GB RAM, and a 1,450 mAh battery. It comes with a 3.2-megapixel rear camera and a 0.3-megapixel front camera. The phone also comes with 8GB of internal storage, which can be expanded by another 32GB using a microSD card. … Surprisingly for its price, the Freedom 251 will run on Android Lollipop 5.1 and comes pre-installed with several apps.

Freedom-251

  • The harsh reality for most OEMs exhibiting their new wares in Barcelona, though, is that users are not finding enough reason to update their smartphones any more:

  • It’s not clear if this flexible screen prototype developed at Queen’s University Human Media Lab on an LG OLED screen will be present at MWC.  It offers a glimpse into the future of mobile device form factor and physical interaction model:

This Flexible Prototype Is a Glimpse at Your Future Phone

The Internet of Things and Wearables

  • “Sense” is a curious-looking intelligent, secure hub for the smart home developed by a team of ex-Mozilla engineers.   It does the usual things you might expect from any such first world proposition in the AI era with a distinct twist:

All the machine learning, computation and data processing takes place on the device itself – not the cloud. The results of that processing – for instance, a video clip of an intruder – is sent to the cloud service in an encrypted format. The keys to decrypt the content are only on the device, meaning even if Sense’s servers were hacked, the attacker wouldn’t gain access to your personal data.

  • Swiss watches overhauled by smart watches:

  • Zzzoomba:

Cloud

Software Engineering

“the easiest code to delete is the code you avoided writing in the first place.”

Remember your neighbor can’t hire a plumber from China but your boss can hire a programmer from a place where the cost/standard of living is far lower.  Always keep an eye on making yourself as valuable as possible to the organization.  (no one is indispensable so don’t delude yourself)

Work and Innovation

Hire promising people — who are in it for the right reasons, not the biggest compensation package — then toss them into the deep end.

  • The 100 coolest people in tech in the UK according to Business Insider makes for a curious list replete with CEOs, Googlers, politicos, journalists and VC types seemingly jockeying for position with actual techies.  Appropriately enough, top of the tree is Eileen Burbidge of Passion Capital, the UK Treasury’s special envoy for Fintech and a special advisor to David Cameron.
  • Accenture will save us once they’ve dealt with the Pride of lions. This from Gadgette editor Holly Brockwell, number 89 on the list:

Society and Culture

  • David Bowie was not one for niceties it seems when it came to turning things down.  The Guardian provided ten examples one of which was the chance of a knighthood.  His response feels very generally applicable:

“I would never have any intention of accepting anything like that.  It’s not what I spent my life working for.”

Header essay dusty walking up hill cropped

Week 11

$
0
0

The Meaning of AlphaGo

Following the heights scaled by AlphaGo last week has illustrated the opposite, the depths of which human beings are capable in the indiscriminate attacks in Belgium with its own horrible and twisted logic:

Merely killing passers-by serves no warlike purpose in itself. The explosive force derives from our reaction to it, from the public attention awarded to it and from the response of the political community.

There’s little to be gained by adding further to the mass of articles on why it was done and what can be done about it.  It is however worth reflecting upon the events and considering whether the concerns that some commentators have about the ability for artificial intelligence to impact the course of human evolution are overplaying the human hand.  Perhaps there are some aspects to human behaviour we could do with a greater sense of urgency on addressing and which we are in danger of bestowing upon our creations as this week’s unfortunate incident with Microsoft’s “racist” AI revealed only too plainly:

This blog is going to continue to focus on our collective achievements with science and technology. Following on from the famous victory over Lee Sedol, Demis Hassabis published a blog post on “what we learned in Seoul with AlphaGo” .  Relatedly, another must-read post included an interview with the legendary Geoff Hinton, the “godfather of neural networks” and creator of backpropagation also now at Google (of course) musing on progress, timescales and obstacles:

My belief is that we’re not going to get human-level abilities until we have systems that have the same number of parameters in them as the brain. So in the brain, you have connections between the neurons called synapses, and they can change. All your knowledge is stored in those synapses. You have about 1,000-trillion synapses—10 to the 15, it’s a very big number. So that’s quite unlike the neural networks we have right now. They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain.

The kind of Deep Learning technology that helped turn AlphaGo into a grandmaster albeit using hypercharged training and huge compute capacity is still rather obtruse for the layman to access but that is likely to change in the coming years.  Key steps to mainstreaming the tech were arguably made this week with the announcement by Google of a user-friendly set of “Cloud Learning” APIs and (crucially) pre-trained models for speech recognition etc are to be made available for general purpose use within Google’s Cloud Platform (GCP):

Further evidence that Google are taking their Machine Learning responsibilities seriously is clearly evident in the decision to stop pursuing humanoid robot development and stick to software instead.

The consequence of all this advance is that the universal basic income (UBI) argument is going to become an ever more important in the coming years.  This Medium post suggests it will emerge as mainstream once we have learned the “lesson of our lives”:

No nation is yet ready for the changes ahead. High labor force non-participation leads to social instability, and a lack of consumers within consumer economies leads to economic instability. So let’s ask ourselves, what’s the purpose of the technologies we’re creating? What’s the purpose of a car that can drive for us, or artificial intelligence that can shoulder 60% of our workload? Is it to allow us to work more hours for even less pay? Or is it to enable us to choose how we work, and to decline any pay/hours we deem insufficient because we’re already earning the incomes that machines aren’t? … What’s the big lesson to learn, in a century when machines can learn?  I offer it’s that jobs are for machines, and life is for people.

Life in the post-work economy will need to address those who wish to cling to an increasingly shaky capitalist hegemony that will likely remain implacably opposed to anything approaching UBI which could end up framing the primary political battleground of the next generation:

The fact is that capitalism—with its tendency to income inequality, information monopolies, and financial power—is running out of steam. It’s time to start thinking about something new.

The Internet of Things and Robots

husqvarna-automower-connect-apple-watch-on-arm-24-HR-770x285

Now Fitbit users can hear praise for their daily activity from a human voice (and in full sentences), though a new integration with Amazon Echo’s digital assistant Alexa.

Android Wear is poised to become the only viable OS not just for the fashion industry’s smartwatches, but for the entire fashion industry at large.

Apps and Services

  • How the Dead Live on in Facebook and will inevitably outnumber the living one day.  When they do it will have profound and disturbing consequences for those left behind who risk mental confusion attempting to distinguish between reality and fiction:

The numbers of the dead on Facebook are growing fast. By 2012, just eight years after the platform was launched, 30 million users with Facebook accounts had died. That number has only gone up since. Some estimates claim more than 8,000 users die each day.

In Facebook, all places are present, all times are now. [The dead exist] in this medium just as I do. … There’s no moving on without any of the millions of dead Facebook users.  ,,, As of yet, there’s no good solution to the problem of dead data, of digital ghosts. The only hope is that the internet’s memory will at some point begin to fade.

The social network is changing how we experience death (Credit:Getty Images)

  • TechCrunch have released a new AI-powered newsbot on Telegram powered by the Chatfuel platform.  It comes across as a little limited in terms of scope since it only takes articles from one source.  A more catholic version would be interesting to see and would make this blog easier to pull together:

TechCrunch Telegram bot in action

Cloud Computing

Cloud Shell session

Devices and Manufacturers

We aren’t done with the mobile revolution. But we are mostly done with it in the developed world.  … the thing that is particularly exciting about new services in the developing world is that they may come with fundamentally new business models. And, it turns out, new business models are even more disruptive than new technologies.

XiaomiMi5

Autonomous Vehicles

  • And right on cue, an excellent blog post on hacking a Tesla S electric car reveals it to be an “enterprise computing system on wheels” which incorporates some really good design decisions notably the inclusion of a first class FOTA update mechanism as well as separation of security domains:

Some reverse engineering of this service showed that the Tesla Model S does NOT seem to send raw CAN frames from the infotainment system to the vehicle. Instead, there is a Vehicle API (VAPI) whereby the CID asks the gateway to perform any one of an “allowed” set of actions.

TH-1

Security

PINMadness

hacked_website_how_compromised

Software Engineering

  • The Stack Overflow 2016 developer survey is essential reading for anyone who works in tech. In amongst the surfeit of coding language breakdowns is this shocking stat demonstrating the “other” technological Singularity the tech industry faces, one based on gender:

GenderInTech

pypi-chart-1

  • Data structures in C seems to betray the influence and impact of of dynamically-typed languages.

China

  • The Makers of Modern China:

Startups and Work

none of those companies had signed contracts, merely “letters of intent”, which did not commit them to anything. One senior figure in the company told me that young inexperienced sales staff were rewarded with a £2,000 bonus every time one of these letters was signed “so they weren’t particularly concerned about the quality of the deal”.

Trump

systemic, global, and amorphous problems … are solvable, but not easily, and not immediately. It is much easier to find someone to blame, which is why we are seeing the return to acceptability of xenophobia and jingoism. Trump on Muslims and Hispanics. Putin on Ukrainians. Erdoğan on Kurds. And the list goes on. The slope from here to scapegoating all dissenters as “enemies of the state” is proving frighteningly slippery.

  • The WashPo transcript of Donald Trump’s meeting with their editorial board is something of a car crash.  It reveals Trump to be extraordinarily fixated with perceived slights and negative reporting at the hands of journalists.  He appears to be more interested in focussing on that than ISIS.  He also seems to change his mind and blow hot, cold and everything in between (or more charitably be very unclear about the detail of his position) multiple times within the interview sometimes within the same response.

In the episode, Bart Simpson gets a glimpse of his future only to find that he’s pretty much a loser, while his sister Lisa has become the president of the US. She calls a meeting in the Oval Office to assess the damage done by her predecessor, Trump; an advisor holds up an plummeting line chart and explains, “We’re broke!”  … The idea of a Trump presidency came about when the writers needed to invent a world in which “everything went as bad as it possibly could”.

Society and Culture

Last year, frustrated, sixth-grader Madeline Messer analyzed 50 popular video games and found that 98% came with built-in boy characters, compared to only 46% that offered girl characters.

Week 15

$
0
0

The Human in the Machine

The role of Human-Assisted AI (or HAAI) in supporting human and machine intelligence for service delivery is a topic that has been covered at length already in this blog in 2016.  A recent ComputerWorld article refers to it as “human in the loop computing”:

If the confidence score is below a certain value, it sends the data to a human annotator to make a judgment. That new human judgment is used both for the business process and is fed back into the machine learning algorithm to make it smarter. In other words, when the machine isn’t sure what the answer is, it relies on a human, then adds that human judgment to its model.

People rather than algorithms remain the often unhappy grunt force behind pretty much all “AI bot” propositions today as this excellent recent Bloomberg post revealingly entitled “the humans hiding behind the chatbots points out:

A handful of companies employ humans pretending to be robots pretending to be humans. In the past two years, companies offering do-anything concierges (Magic, Facebook’s M, GoButler); shopping assistants (Operator, Mezi); and e-mail schedulers (X.ai, Clara) have sprung up. The goal for most of these businesses is to require as few humans as possible. People are expensive. They don’t scale. They need health insurance. But for now, the companies are largely powered by people, clicking behind the curtain and making it look like magic.

The “Wizard of Oz” pattern is an approach that was pioneered in the UK in the Noughties by text question and answer service AQA and voice to text service provider SpinVox.  AQA were relatively transparent about the humans in their SMS loop.  SpinVox, however, were notoriously more secretive about their ‘advanced’ AI technology (called ‘D2’) until they were eventually unmasked.  AI even back then was a big draw and their story provides a cautionary tale for today’s HAAI aspirants:

SpinVox

Wind on a few years to the present and the “app as assistant” paradigm has re-emerged as the next big thing.  Bot hype is everywhere with VCs amongst the most bullish in their pronouncements.  There’s no doubt the AI this time around is far more sophisticated and powerful:

Products like Fin/Operator/M are at least partially powered by real humans. Other important ingredients are NLP to process text inputs,CV and deep learning improvements in audio-to-text to process inputs like audio/photos/video, and more. Its so much more than text and AI.

Even so, it’s not clear we really know understand how to leverage techniques like Deep Learning to build Conversational UIs yet:

Truly ‘conversational’ software can manage a complex, evolving state. It can use knowledge provided a priori by the developer, and can gain new knowledge from users. That’s a difficult problem to solve.

It’s unlikely the typical chatbot startup can make much progress in this territory without serious PhD level research chops on board in niche techniques most of us have never heard of, like DMN:

AskMeAnything

Or indeed, if they get there, that IM chat-based messaging is the right interface anyway.  Dan Grover, a Tencent employee and WeChat product manager based in Guangzhou China, has published an essential read on how the WeChat service interface owes a lot of its success to integrated calls to action in the form of lists and other buttons as well as offering a unified platform interface to features such as payments.  It’s an approach being adopted by Telegram bots and within Facebook to work around mobile platform shortcomings:

ChinaSouthern

BotAffordances

It’s worth reflecting on these two considerations by way of backdrop next time you read yet more hype of Facebook’s M virtual assistant that “lives inside of Messenger”:

  • The degree of dependence on an underlying Mechanical Turk model.  To what extent is it humans all the way down?
  • The possibility that conversational UI isn’t the right paradigm for a rich service interface anyway.

Messenger and M integration is clearly a huge deal to Facebook. Reading between the lines, however, it is still very much a work in progress with seamless mobile shopping a particular area of focus. Early feedback from Gizmodo on the handful of M chatbots themselves is perhaps unsurprisingly disappointing highlighting a number of characteristic limitations such as  global amnesia and long uncomfortable pauses.  It doesn’t make for a very human experience.

Facebook Chatbots Are Frustrating and Useless

Poncho is basically looking for one particular phrase, and that’s “Do I need [insert item]?” As you can see in the screenshots above, referring to my past chats or previous answers doesn’t work (you know, like a normal conversation). Every chat with Poncho exists in a vacuum and it forces you into a robotic back and forth, no matter how Poncho disguises it—be it with funny GIFs or pop culture references.

Google and Android

Android has become the Windows of the mobile world, serving as the ubiquitous operating system of choice, and Google’s software licensing terms are leading device manufacturers to bundle and promote its apps ahead of others. Another commonality is that Google was one of the behind-the-scenes agitators during Microsoft’s IE tribulations, and Microsoft returned the favor in 2013 when it petitioned the EC to look into Android app bundling.

GoogleVisits

AndroidShareApril2016

Big Companies, Big Mistakes

  • With Google re-enacting Microsoft history, it’s worth reflecting on what a still Microsoft-dominated world of 2007 was like. Steve Ballmer laughing at the iPhone v1.0 “the most expensive phone in the world” should be required viewing for industry executives who think they know the business better than their customers. This is literally what disruptive innovation from above looks like for incumbents:

Intel turned down an opportunity to provide the processor for the iPhone, believing that Apple was unlikely to sell enough of them to justify the development costs.

Intel will lay off 11% of its global workforce, up to 12,000 employees, a painful downsizing aimed at accelerating its shift away from the waning PC market to one more focused on cloud computing and connected devices.

  • Apple themselves seem to have fallen into an internal culture clash not dissimilar to that which afflicted Nokia in it’s pomp when different teams expended their energies on each other rather than external threats.  Presumably Apple management are far too wise to history to let the sort of wrangling apparently happening within their cloud computing division from dragging on much longer.

Artificial Intelligence

NeuralNetTF

“the escalator of reason leads societies to greater benevolence regardless of species origin. A.I.s will have to step on the escalator of reason just like humans have, because they will need to bargain for goods in a human-dominated economy and they will face human resistance to bad behavior.”

tu hao =
土 – earth, peasant +
豪 – bold and unconstrained (or grand, heroic, fine)

robot_maids2

Security

  • Good CESG article exposing the spurious rigour involved in enforcing regular password expiry:

The problem is that this doesn’t take into account the inconvenience to users  – the ‘usability costs’ – of forcing users to frequently change their passwords. The majority of password policies force us to use passwords that we find hard to remember. Our passwords have to be as long as possible and as ‘random’ as possible. And while we can manage this for a handful of passwords, we can’t do this for the dozens of passwords we now use in our online lives.

The Internet of Things

  • On IoT consortia:

  • Another day, another dodgy IoT use case.  This is the sort of thing that gives IoT a bad press amongst ordinary consumers:

Manufacturers and Devices

  • Amazing MIT shapeshifting display:

  • InFocus is Foxconn’s consumer smartphone brand.  Here’s a review of the sub-£80 (at least in some places online) InFocus 4G model the M560.  It’s hard to compete with these specs at that price:

Infocus-M560

Software and Programming

Jaspy Screencast

On the art of Fixing Things

  • This week’s episode of The Reassembler has ex-Top Gear host James May reconstructing a completely disassembled electric guitar over six hours.  It’s as much fun listening to him talking about the history of different screwdriver types as it is following the instrument rebuilding.
  • The Restart Project just released a short video about another large scale problem facing humanity – electronic waste.  They too encourage a reuse and repair philosophy and run workshops to help those .   A key learning is to avoid panic and overreaction when Things Fall Apart and instead learn to stay calm and think through the issues.  It’s an approach that serves equally well with software.

Science

DNA can theoretically store one exabyte of data in a volume of a grain of sand. That’s roughly eight magnitudes denser than that of tape, and roughly equivalent to 200 million DVDs. … What’s more, DNA is extremely stable. The average lifespan for rotating disks and optical storage systems is at most a few decades.  In contrast, DNA has the potential to reliably store data for centuries without significant decay.

  • Aurora Borealis and Australis viewed from the ISS:

Culture and Society

instead of outgrowing traditional capitalism, technology is going to erode the base of capitalism in the next few decades. Rather than building on top of a strong capitalist base, we will be dismantling capitalism as we build something new.

The Coming End of Capitalism

  • The mess that is the Middle East:

Work and Play

  • HBR have just published an article entitled “Embracing Agile” which represents something of a watershed.  Although Agile development is an approach that has long been championed in IT and software, it has arguably never been considered relevant for reinventing the wider business environment. Key ingredients in the mix they identify include a structured process built around multidisciplinary self-governing teams, stack-ranking tasks for sprints, a clear definition of done and above all, full endorsement and support from the very top.

it’s extremely unlikely you can greatly improve your reading speed without missing out on a lot of meaning. Reading is about language comprehension, not visual ability.

Elon Musk Résumé

Hardly anyone has gotten a significant raise since the recession. Gen Xers are worth singling out because they’re in their peak earning years. They’ve now missed out on what should be the biggest raises of their careers.

GenXWoe

Since its release seven years ago, Minecraft has become a global sensation, captivating a generation of children. There are over 100 million registered players, and it’s now the third-best-­selling video game in history, after Tetris and Wii Sports.

  • It is at heart ‘an incredibly complex game’ that encourages computational thinking and fluency with the command line that are both key to software engineering later in life.  The author relates this illustrative anecdote:

One day last fall, I visited Gus, a seventh ­grader in Brooklyn.  … I watched as he typed a command to endow himself with a better weapon: “/give AdventureNerd bow 1 0 {Unbreakable:1,ench:[{id:51,lvl:1}],display:{Name:“Destiny”}}.” What the command did was give a bow-­and-­arrow weapon to AdventureNerd, Gus’s avatar; make the bow unbreakable; endow it with magic; and name the weapon Destiny, displayed in a tag floating over the weapon.

  • Here’s something equally inscrutable.  Steve Davis, forever associated with anyone of a certain age with almost total dominance of the world of snooker in the 80’s, is reinventing himself as a techno DJ:

Prince

  • The news that another 80’s rock superstar had died took a while to sink in and then generated an inevitable outpouring of shock and sadness.   He left behind an incredible back catalogue and by some accounts 26 albums worth of unreleased material.  One suspects his well-documented troubles with Warner Brothers will rumble on after his death with the amount of money they could make from him now he’s dead.  And so the dead live on without recourse to cryogenics.
  • Musical highlights abound of course with several key live performances available in various degrees of graininess on YouTube if not as Spotify tracks.  His extraordinary strange performance of Radiohead’s Creep at Coachella and celebrated rendition of Purple Rain at the SuperBowl help underline what an amazing live performer he was in an era when genuine musical ability really mattered.  The virtuoso guitar showmanship comes across well here:

  • If you have any remaining doubt as to his rock credentials, do yourself a favour and go 3:28 seconds into the ‘supergroup’ rendition of ‘While My Guitar Gently Weeps’ below performed in celebration of the late George Harrison 12 years ago in 2004. He totally owns the next 3 minutes from there to his Hendrix-like flourish at the end.  As the man put it in Around the World in a Day (1985):

“I can be a weirder fey psychedelic badass mushroom than you, Paul McCartney.”

  • There’s a distinct irony here.  Prince doubtless respected the Beatles and Harrison in particular as a fellow guitar genius but he also saw them as representatives of a white rock scene that routinely marginalizes black rock musicians and puts them in a separate category when in fact it’s their scene that has been appropriated by the majority:

When rock music is defined by and limited to white people, only white people get to fit in the critically acclaimed pantheon of rock geniuses. Black performers are made to seem an exception in the genre they invented. … Prince came from a long line of eccentric rock geniuses, and part of how he honored them was by forcing even the most obtuse listener to understand that rock was, and always has been, black music.

Viewing all 69 articles
Browse latest View live


Latest Images