MyBKV - Infoman

Posts by Infoman

Infoman

The unnecessary demise of Barnes and Noble

458 spiralpolitik 1 day 323

http://audreyii-fic.tumblr.com/post/170886347853/the-entirely-unnecessary-demise-of-barnes-noble

news.ycombinator.com/item?id=16389342

“Whether the Andrea Gail rolls, pitch-poles, or gets driven down, she winds up, one way or another, in a position from which she cannot recover. Among marine architects this is known as the zero-moment point – the point of no return.” –Sebastian Junger, “The Perfect Storm”

Posts like this aren’t my usual fare, but there’s a lot of readers on Tumblr. So y’all might be interested – or, if not, you really should be.

On Monday, this went down:

image image That’s the bloodless, matter-of-fact, ho-hum business event way of describing it. Let me paint you a different picture.

On Monday morning, every single Barnes & Noble location – that’s 781 stores – told their full-time employees to pack up and leave. The eliminated positions were as follows: the head cashiers (those are the people responsible for handling the money), the receiving managers (the people responsible for bringing in product and making sure it goes where it should), the digital leads (the people responsible for solving Nook problems), the newsstand leads (the people responsible for distributing the magazines), and the bargain leads (the people responsible for keeping up the massive discount sections). A few of the larger stores were able to spare their head cashiers and their receiving managers, but not many.

Just about everyone lost between 3 and 7 employees. The unofficial numbers put the total around 1,800 people.

People.

image image We’re not talking post-holiday culling of seasonal workers. This was the Red Wedding. Every person laid off was a full-time employee. These were people for whom Barnes & Noble was a career. Most of them had given 5, 10, 20 years to the company. In most cases it was their sole source of income.

There was no warning.

But it gets worse.

The people who lost their jobs had been actively assured this would NOT happen for the past several months. Home Office decided last year that these positions – head cashiers, receiving managers, leads – were due to be eliminated… but no layoffs were to take place. All current employees were to be grandfathered in. The positions wouldn’t go away until the people currently holding them chose to leave.

For months they told everyone this.

Then on Monday, each person was called into the manager’s office. Fifteen minutes later, each person gathered up their things and left.

image Severance packages varied; usually as little as two weeks pay. Demotions were not offered. Those laid off – and I’d like to reiterate that these were career employees – were offered the opportunity to reapply for part-time positions. At base pay.

“Thanks for your 17 years of service. If you’d like to come back in two months to work registers for minimum wage, we’d love to have you.”

(Don’t let the glamor of book sales fool you. B&N entry pay is minimum wage.)

Now. You’re going to hear a lot about the dropped sales. You’re going to hear a lot about Amazon. You’re going to hear about how there was no way around this.

That, dear friends, is bullshit.

image image I want to talk to you about why sales dropped during the holidays.

A company wants cash on hand to look good to stockholders. The quickest way for a company to get cash on hand is to cut back on payroll. Which Barnes & Noble did this holiday season. During December, staffing at most locations was no different than it would be on your average day in June.

Something kinda important is happening in December, btw.

image In particular, hours in receiving were carved to the bone. You know what that means? It means that product – product that could be selling – sat unopened in boxes. In many cases, those boxes had already been logged into the system. The computers showed we had them. Customers came in, expecting to purchase things, knowing they were in the store! But what they wanted was buried under 100, 200 boxes. And there were no employees to find them. There were barely any employees available at all.

Customers went away annoyed. And they shopped on Amazon instead.

Because, well, why not?

(As a side note: people often want to know why Amazon’s prices are so much lower than B&N’s, and why B&N doesn’t price match. There’s a lot of different reasons, but the biggest is that Amazon loss-leads their books: that is, sells them at a loss, then makes up the money with expensive add-ons, like Echos or Kindles or other non-book stuff.

image Barnes & Noble only has books; they can’t make up the discount loss by selling water coolers. Amazon undercuts prices, and once they’re the only players in the market, that will stop. Just so you know.)

Yeah, sales sucked. Cutting hours during the busiest season of the year punched quarter earnings in the gut. Big shock.

So, if you were trying to solve the problem – if you were trying to revitalize this business – what would you do?

“Oh, I know! I would tighten my belt at the executive level, then I would double-down on what we can offer that Amazon can’t: enthusiastic staff that can find and upsell books to suit each customer, and the largest in-store selection possible so that everyone who comes in can walk away with what they want. If I absolutely had no choice but to reduce payroll, I would eliminate some part-time positions, and count on the knowledge and experience of our veterans to get us through the lean times.”

image Those are definitely the choices you would make if you wanted to rebuild a company to last.

But here’s a secret:

The Barnes & Noble executives do not intend to rebuild.

How do I know this? Because every decision from the upper levels is being made solely to increase cash on hand.

There’s been so many things – so many things – but me tell you about the canary in the coal mine. Let me tell you how I know saving Barnes & Noble is not in the home office’s plans.

Last summer, the decision was made to switch to “ship from store”. Previously, when a customer ordered a book online, the book would be shipped to them from one of our warehouses. The new policy, however, had stores taking books off their shelves, packaging them up, and sending them out each day.

This was to “decrease shipping time by sending books from the closest location to the customer.”

(Spoiler alert: that wasn’t true.)

So each store takes employees off the selling floor – where they could, you know, help customers – and sets them to fill orders. The stores remove books from their own shelves and mail them out.

The stores do not get credit for those sales.

Let me repeat:

The stores do not get credit for those sales.

The company makes money. The brick and mortar store – which Barnes & Noble is based on – loses the opportunity to sell that book (pissing off customers), and gets nothing in return.

Which hurts the bottom line of that store.

“Uh-oh, your sales dropped. Better cut back hours.”

This is a decision that is only made if the executive level of a company is no longer interested in helping their business. This is a decision that is made only if the executive level has decided the company is dying, and don’t care if they hasten along the demise as long as they can harvest the organs for themselves and leave everyone else with the shriveled husk.

Speaking of organ harvesters.

image By the way, it should be noted that the last CEO, who worked for Barnes & Noble for less than a year, received a $4,500,000 payout. The CEO before him, who also worked at Barnes & Noble for less than a year, received a $10,000,000 payout.

image The company saved $40m by firing 1,800 employees.

After paying out $14,500,000 to two executives.

Which brings us back around to Monday’s layoffs.

By getting rid of their most expensive (ie, most experienced) workers, B&N was able to replace said workers with part-time, benefit-less, minimum-wage employees. This is not to knock newbies – we were all ones once! – but a new hire can’t do what a 15 year veteran can. They just can’t. Not right away. Not for a long time. And not in the specialized departments that were laid off: head cashiering, receiving, digital, newsstand, bargain. Those are hard jobs that take training.

Who’s going to train?

And who’s going to want to be trained for something that hard, when there’s no possibility of a promotion down the line? When you’re only working minimum wage for a maximum of 25 hours a week?

At the beginning of this post, I quoted Junger and the zero-moment point. The loss of these veterans, and the positions they worked, cannot be recovered from. Barnes & Noble is going under.

image “That sucks, but it’s the way capitalism works. Hate the game, not the player.”

Sure. Maybe.

But this.

Except this.

On Monday morning, as thousands of lives were upended, Barnes & Noble – literally in the same hour – released this:

image image “His deep knowledge of retail and proven track record are exactly what we need to invigorate our merchandising strategy and grow our business.”

Grow our business.

image I don’t know what happens after Barnes & Noble sinks. It’s all well and good to say “Support indie stores!” but there are huge swaths of America where there aren’t any. B&N is the last thing standing between Amazon and a total monopoly of the publishing industry, and a monopoly is never a good thing. But the entire book world needs to be prepared, because it’s coming.

The zero-moment point came Monday, and in the crassest, cruelest, more heartless way possible.

Barnes & Noble has slit its own wrists. Now we just wait to bleed out.

2018/02/17
Infoman

Missing data hinder replication of artificial intelligence

studies 47 jonbaer 10 hrs 12

http://www.sciencemag.org/news/2018/02/missing-data-hinder-replication-artificial-intelligence-studies

news.ycombinator.com/item?id=16392517

Last year, computer scientists at the University of Montreal (U of M) in Canada were eager to show off a new speech recognition algorithm, and they wanted to compare it to a benchmark, an algorithm from a well-known scientist. The only problem: The benchmark's source code wasn't published. The researchers had to recreate it from the published description. But they couldn't get their version to match the benchmark's claimed performance, says Nan Rosemary Ke, a Ph.D. student in the U of M lab. "We tried for 2 months and we couldn't get anywhere close."

The booming field of artificial intelligence (AI) is grappling with a replication crisis, much like the ones that have afflicted psychology, medicine, and other fields over the past decade. AI researchers have found it difficult to reproduce many key results, and that is leading to a new conscientiousness about research methods and publication protocols. "I think people outside the field might assume that because we have code, reproducibility is kind of guaranteed," says Nicolas Rougier, a computational neuroscientist at France's National Institute for Research in Computer Science and Automation in Bordeaux. "Far from it." Last week, at a meeting of the Association for the Advancement of Artificial Intelligence (AAAI) in New Orleans, Louisiana, reproducibility was on the agenda, with some teams diagnosing the problem—and one laying out tools to mitigate it.

The most basic problem is that researchers often don't share their source code. At the AAAI meeting, Odd Erik Gundersen, a computer scientist at the Norwegian University of Science and Technology in Trondheim, reported the results of a survey of 400 algorithms presented in papers at two top AI conferences in the past few years. He found that only 6% of the presenters shared the algorithm's code. Only a third shared the data they tested their algorithms on, and just half shared "pseudocode"—a limited summary of an algorithm. (In many cases, code is also absent from AI papers published in journals, including Science and Nature.)

Researchers say there are many reasons for the missing details: The code might be a work in progress, owned by a company, or held tightly by a researcher eager to stay ahead of the competition. It might be dependent on other code, itself unpublished. Or it might be that the code is simply lost, on a crashed disk or stolen laptop—what Rougier calls the "my dog ate my program" problem.

Assuming you can get and run the original code, it still might not do what you expect. In the area of AI called machine learning, in which computers derive expertise from experience, the training data for an algorithm can influence its performance. Ke suspects that not knowing the training for the speech-recognition benchmark was what tripped up her group. "There's randomness from one run to another," she says. You can get "really, really lucky and have one run with a really good number," she adds. "That's usually what people report."

In a survey of 400 artificial intelligence papers presented at major conferences, just 6% included code for the papers' algorithms. Some 30% included test data, whereas 54% included pseudocode, a limited summary of an algorithm.

CREDITS: (GRAPHIC) E. HAND/SCIENCE; (DATA) GUNDERSEN AND KJENSMO, ASSOCIATION FOR THE ADVANCEMENT OF ARTIFICIAL INTELLIGENCE 2018

At the AAAI meeting, Peter Henderson, a computer scientist at McGill University in Montreal, showed that the performance of AIs designed to learn by trial and error is highly sensitive not only to the exact code used, but also to the random numbers generated to kick off training, and to "hyperparameters"—settings that are not core to the algorithm but that affect how quickly it learns. He ran several of these "reinforcement learning" algorithms under different conditions and found wildly different results. For example, a virtual "half-cheetah"—a stick figure used in motion algorithms—could learn to sprint in one test but would flail around on the floor in another. Henderson says researchers should document more of these key details. "We're trying to push the field to have better experimental procedures, better evaluation methods," he says.

Henderson's experiment was conducted in a test bed for reinforcement learning algorithms called Gym, created by OpenAI, a nonprofit based in San Francisco, California. John Schulman, a computer scientist at OpenAI who helped create Gym, says that it helps standardize experiments. "Before Gym, a lot of people were working on reinforcement learning, but everyone kind of cooked up their own environments for their experiments, and that made it hard to compare results across papers," he says.

IBM Research presented another tool at the AAAI meeting to aid replication: a system for recreating unpublished source code automatically, saving researchers days or weeks of effort. It's a neural network—a machine learning algorithm made of layers of small computational units, analogous to neurons—that is designed to recreate other neural networks. It scans an AI research paper looking for a chart or diagram describing a neural net, parses those data into layers and connections, and generates the network in new code. The tool has now reproduced hundreds of published neural networks, and IBM is planning to make them available in an open, online repository.

Joaquin Vanschoren, a computer scientist at Eindhoven University of Technology in the Netherlands, has created another repository for would-be replicators: a website called OpenML. It hosts not only algorithms, but also data sets and more than 8 million experimental runs with all their attendant details. "The exact way that you run your experiments is full of undocumented assumptions and decisions," Vanschoren says. "A lot of this detail never makes it into papers."

Psychology has dealt with its reproducibility crisis in part by creating a culture that favors replication, and AI is starting to do the same. In 2015, Rougier helped start ReScience, a computer science journal dedicated to replications. The large Neural Information Processing Systems conference has started linking from its website to papers' source code when available. And Ke is helping organize a "reproducibility challenge," in which researchers are invited to try to replicate papers submitted for an upcoming conference. Ke says nearly 100 replications are in progress, mostly by students, who may receive academic credit for their efforts.

Yet AI researchers say the incentives are still not aligned with reproducibility. They don't have time to test algorithms under every condition, or the space in articles to document every hyperparameter they tried. They feel pressure to publish quickly, given that many papers are posted online to arXiv every day without peer review. And many are reluctant to report failed replications. At ReScience, for example, all the published replications have so far been positive. Rougier says he's been told of failed attempts, but young researchers often don't want to be seen as criticizing senior researchers. That's one reason why Ke declined to name the researcher behind the speech recognition algorithm she wanted to use as a benchmark.

Gundersen says the culture needs to change. "It's not about shaming," he says. "It's just about being honest."

2018/02/17
Infoman

The Benjamin Franklin method for learning more from programming books

365 nancyhua 11 hrs 75

http://www.pathsensitive.com/2018/01/the-benjamin-franklin-method-of-reading.html

news.ycombinator.com/item?id=16390046

Let’s face it, programming books suck. Those general books on distributed systems or data science or whatever can be tomes for a lifetime, but, with few exceptions, there’s something about the books on how to write code in a language/framework/database/cupcake-maker, the ones with the animal covers and the cutesy sample apps, they just tend to be so forgettable, so trite, so….uneducational.

I think I’ve figured out why I don’t like them, and it’s not just that they teach skills rapidly approaching expiration. It’s their pedagogical approach. The teaching algorithm seems to be: write these programs where we’ve told you everything to do, and you’ll come out knowing this language/framework/database/cupcake-maker. Central to these books are the long code listings for the reader to reproduce. Here’s an example, from one of the better books in this category

class User < ApplicationRecord attr_accessor :remember_token before_save { self.email = email.downcase } validates :name, presence: true, length: { maximum: 50 } VALID_EMAIL_REGEX = /\A[\w+-.]+@[a-z\d-.]+.[a-z]+\z/i validates :email, presence: true, length: { maximum: 255 }, format: { with: VALID_EMAIL_REGEX }, uniqueness: { case_sensitive: false } has_secure_password validates :password, presence: true, length: { minimum: 6 } # …another 30 lines follows... end Traditionally, there are two ways to study a page like this:

Type out every line of code Copy+paste the code from their website, maybe play around and make small changes Approach #1 is a method that, like a lecture, causes the code to go from the author’s page to the reader’s screen without passing through the heads of either. The second is like trying to learn how to make a car by taking apart a seatbelt and stereo: you’re just toying with small pieces. Neither is a sound way to learn.

If you had an expert tutor, they wouldn’t teach you by handing you a page of code. Still, these books are what we have. How can we read them in a way that follows the principles of learning? Read on.

Mental Representations

According to K. Anders Ericsson in his book Peak, expertise is a process of building mental representations. We can see this because expert minds store knowledge in a compressed fashion. Musicians can memorize a page of music far faster than a page of random notes. Expert chess players told to memorize a board position will do much better than amateurs, but, when they make a mistake, they’ll misplace whole groups of pieces.

This is possible because music and chess positions have structure that makes them look very different from a page of random notes or a random permutation of pieces. Technically speaking, they have lower perplexity than random noise. So, even though there are 26 letters in the English alphabet, Claude Shannon showed that the information content of English is about 1 bit per letter: given a random prefix of a paragraph, people can guess the next letter about half the time.

This is why a programmer skilled in a technology can look at code using it and read through it like fiction, only pausing at the surprising bits, while the novice is laboring line-by-line. This is also why a smart code-completion tool can guess a long sequence of code from the first couple lines. With a better mental representation, understanding code is simply less work.

(How do these mental representations work? My officemate Zenna Tavares argues they are distribution-sensitive data structures.)

This is exactly what’s missing from the “just type out the code” approach: there’s nothing forcing your mind to represent the program as anything better than a sequence of characters. Yet being able to force your mind to do this would mean being able to learn concepts more rapidly. Here’s a 200 year-old idea for doing so.

The Benjamin Franklin Method

I don’t know what’s more impressive: that Benjamin Franklin became a luminary in everything from politics to physics, or that he did this without modern educational techniques such as schools, teachers, or StackOverflow. As part of this, he discovered a powerful method of self-study. I’ll let him speak for himself (or go read someone else’s summary).

About this time I met with an odd volume of the Spectator. It was the third. I had never before seen any of them. I bought it, read it over and over, and was much delighted with it. I thought the writing excellent, and wished, if possible, to imitate it. With this view I took some of the papers, and, making short hints of the sentiment in each sentence, laid them by a few days, and then, without looking at the book, try'd to compleat the papers again, by expressing each hinted sentiment at length, and as fully as it had been expressed before, in any suitable words that should come to hand. Then I compared my Spectator with the original, discovered some of my faults, and corrected them.

—Benjamin Franklin, Autobiography This process is a little bit like being a human autoencoder. An autoencoder is a neural network that tries to produce output the same as its input, but passing through an intermediate layer which is too small to fully represent the data. In doing so, it’s forced to learn a more compact representation. Here, the neural net in question is that den of dendrons in your head.

K. Anders Ericsson likens it to how artists practice by trying to imitate some famous work. Mathematicians are taught to attempt to prove most theorems themselves when reading a book or paper --- even if they can’t, they’ll have an easier time compressing the proof to its basic insight. I used this process to get a better eye for graphical design; it was like LASIK.

But the basic version idea applied to programming books is particularly simple yet effective.

Here’s how it works:

Read your programming book as normal. When you get to a code sample, read it over

Then close the book.

Then try to type it up.

Simple, right? But try it and watch as you’re forced to learn some of the structure of the code.

It’s a lot like the way you may have already been doing it, just with more learning.

Acknowledgments

Thanks to Andrew Sheng and Billy Moses for comments on previous drafts of this post.

2018/02/16
Infoman

Building Your Own CDN for Fun and Profit

177 janoszen 10 hrs 47

https://pasztor.at/blog/building-your-own-cdn

news.ycombinator.com/item?id=16374645

As you can (hopefully) see from this site, I like my pages fast. Very, very fast. Now, before we jump into this, let me be very clear about it: using a CDN will only get you so far. If your site is slow because of shoddy frontend work, a CDN isn’t going to help you much. You need to get your frontend work right first. However, once you’ve optimized everything you could, it’s time to look at content delivery.

My main problem was that even though you could get the inital website load with a single HTTP request, my server being hosted in Frankfurt, the folks from Australia still had to wait up to 2-3 seconds to get it. Round trip times of over 300 ms and a lot of providers inbetween made the page load just like any other Wordpress page.

So what can we do about it? One solution, of course, would be the use of a traditional CDN. However, most commercial CDNs pull the data from your server on request and then cache it for a while.

PlantUML SVG diagram

However, the initial page load is slower with a CDN than without it, since the CDN is a slight detour for the content. This is not a problem if you have a high traffic site since the content stays in the cache all the time. If, on the other hand, you are running a small blog like I do, the content drops out of the cache pretty much all the time. So, in effect, a traditional pull-CDN would make this site slower. I could, of course, use a push-CDN where I can upload the content directly, but those seem to be quite pricey in comparison to what I’m about to build.

How do CDNs work?

Our plan is clear: on our path to world domination we need to make our content available everywhere fast. That means our content needs to be close to the audience. Conveniently, there are a lot of cloud providers that offer cheap virtual servers in multiple regions. We can just put our content on, say, 6 servers and we’re good, right?

Well, not so fast. How is the user going to be routed to the right server? Let’s take a look at the process of actually getting a site. First, the users browser uses the Domain Name System (DNS) to look up the IP address of the website. Once it has the IP address, it can connect the website and download the requested page.

PlantUML SVG diagram

If we think about it as simple as this, the solution is quite simple: we need a smart DNS server that does a GeoIP lookup on the requesting IP address and returns the IP address closest to it. And indeed, that’s (almost) how commercial CDNs do it. There is a bit more engineering involved, like measuring latencies, but this is basically how it’s done.

Making the DNS servers fast

Now the next question arises: how do we make the DNS server fast? Getting the website download to go to the closest node is only half the job, if the DNS lookup has to go all the way around the planet, that’s still a HUGE lag.

As it turns out, the infrastructure underpinning the internet is uniquely suitable to solve this problem. Network providers use the Border Gateway Protocol to tell each other which networks they can reach and how many hops away they are. The end user ISP then, in most cases, takes the shortest route to reach the destination.

If we now advertise the IP addresses in multiple locations, the DNS request will always be routed to the closest node. This is called BGP Anycast.

Why not use BGP Anycast for the website download?

Wait, hold on, if we can do this, why don’t we simply use BGP to route the web traffic? Well, there are three reasons.

First of all, doing BGP Anycast requires control over the network hardware and a pool of at least 256 IP addresses, which is way over our budget.

Second, BGP routes are not that stable. While DNS requests only require a single packet to be sent in both directions, HTTP (web) requests require establishing a connection to download the content. If the route changes, the HTTP connection is broken.

And finally, the lowest count of hops, which is the basis of BGP route calculations, does not guarantee the lowest round trip time. A hop across the ocean may be just one hop, but it’s a damn long one.

Further reading: Linkedin Engineering has a wonderful blog post about this topic.

Setting up DNS

Since we have established that we can’t run our own BGP Anycast, this means we can also not run our own DNS servers. So let’s go shopping! … OK, as it turns out, DNS providers that offer BGP Anycast servers and latency-based routing are a little hard to come by. During my search I found only two, the rather pricey Dyn and the dirt-cheap Amazon Route53.

Since we are cheap, Route53 it is. We add our domain and then start setting up the IPs for our machines. We need as many DNS records as we have servers around the globe (edge locations), and each record should look like this:

Route53 latency-based routing should be set up in Route53 by creating A records with the IP of the edge location, and then setting the routing policy to "latency". The set ID should be something unique, and the location should be the one closest to our edge location. Tip: it is useful to set up a health check for each of the edge locations so they are removed if they go down.

Distributing content

The next issue we need to tackle is distributing content. Each of your edge nodes needs to have the same content. If you are using a static site generator like Jekyll, your task is easy: simply copy the generated HTML files on all servers. Something as simple as rsync might just do the trick.

If you want to use a content editing system like Wordpress, you have a significantly harder job since it is not built to run on a CDN. It can be done, but it’s not without its drawbacks, and the distribution of static content is still a problem. You may have to create a distributed object storage for that to fully work.

Using SSL/TLS certificates

The next pain point is using SSL/TLS certificates. Actually, let’s call them what they are: x509 certificates. Each of your edge locations needs to have a valid certificate for your domain. The simple solution, of course, is to use LetsEncrypt to generate a different certificate for each, but you have to be careful. LE has a rate limit, which I ran into on one of my edge nodes. In fact, I had to take the London node down for the time being until the weekly limit expires.

However, I am using Traefik as my proxy of choice, which supports using a distributed key-value store or even Apache Zookeeper as the backend for synchronization. While this requires a bit more engineering, it is probably a lot more stable in the long run.

The results

Time for the truth, how does my CDN perform? Using this tool, let’s see some global stats:

Oregon: 246ms, California: 298ms, Ohio: 227ms, Virginia: 108ms, Ireland: 217ms, Frankfurt: 44ms, London: 110ms, Mumbai: 870ms, Singapore: 517ms, Seoul: 253ms, Tokyo: 150ms, Sidney: 358ms, Sao Paulo: 911ms As you can see, the results are pretty decent. I might need two more nodes, one in Asia and one in South America to get better load times there.

Frequently asked questions

When I do projects like this, people usually ask me: “Why do you do this? You must like pain.” Yes, to some extent I like doing things differently just for the sake of exploring new options and technologies, building your own CDN may make a lot of sense. Let’s address some of the questions about this setup.

Let’s be clear: if a commercial provider comes out with an affordable push CDN that allows me to do nice URLs, SSL and custom headers, I’ll absolutely throw money at them and stop running my own infrastructure. As fun as it was to build, I have enough servers to run without this.

Why don’t you just use CloudFlare?

CloudFlare is a wonderful tool for many, but as outlined above, CDNs drop unused content from their cache. On other sites that I’m managing I see a cache rate of about 75% with the correct setup. Having your own CDN means 100% of the content is always in cache, and there are no additional round trips to the origin server.

Why don’t you use S3 and CloudFront?

Amazon S3 has an option to host static websites, and it works in conjunction with CloudFront. However, it does not allow you to set custom headers for caching, nice URLs, etc. For that, you need Lambda@Edge, a tool that lets you run code on the CloudFront edge nodes. Lambda@Edge, however, has the same problem as CDNs: if it doesn’t receive requests for a certain time, the container running it is shut down and needs up to a second to boot up.

Why don’t you use Google AMP?

Google AMP only brings benefits when people visit your site from the Google search engine. My most of my traffic does not come from Google so that won’t solve the problem. So it really only benefits Google, nobody else. Oh, and I’m perfectly capable of building a fast website without the dumbed down HTML they offer.

Who cares? 3 seconds is a wonderful load time!

I’m a DevOps engineer who specializes in delivering content. If anyone, I should have a website that’s fast around the globe, no?

Oh, and I like to flip Google AMP off because it’s a terrible technology. Not that they’d care.

Build your own

Now it’s up to you: do you want to build your own CDN? The source code for mine is right there on my GitHub. Go nuts!

2018/02/14
Infoman

Everything You Know About the Blockchain Is Wrong

5 buffyoda 13 hrs 0

http://degoes.net/articles/blockchain-myths

news.ycombinator.com/item?id=16368993

You don’t understand blockchain.

Well, maybe you do. But if you don’t see what all the fuss is about, wonder why anyone uses blockchain technology instead of Postgres, or think Tor figured out decentralization long before Bitcoin, then I have some news for you:

Every damn thing you know about the blockchain is wrong.

In this post, I going to talk about what I see as the top 8 myths of blockchain. Be prepared for bold claims and new ways of looking at the space.

Myth 1: Blockchain Is Digital Currencies

Many early applications of blockchain technology have been directed at the creation of digital assets that can be used as currencies (more precisely: de-centralized, consensus-driven, append-only ledgers).

Blockchain technology itself, however, is neither about nor restricted to the creation of digital currencies.

In the most general possible sense, blockchain technology refers to a mathematical innovation that allows us to incentivize independent parties in an untrusted, purely consensual network to provide well-defined, agreed upon services.

“Blockchain technology refers to a mathematical innovation that allows us to incentivize independent parties in an untrusted, purely consensual network to provide well-defined, agreed upon services.”

It is hard to overstate just how important this innovation is. One of the oldest problems in the history of civilization is figuring out how to get different parties to work together.

Previously, legal markets have heavily relied on physical force to incentivize independent parties to provide (contractually) agreed upon services. Indeed, this is one of the important functions of governments.

Force works reasonably well for trustworthy parties in the same jurisdiction, but is costly and slow. Force doesn’t work in cases where the parties are untrusted, in cases where the parties span different jurisdictions, or in cases where speed or low costs are critical.

Blockchain technology provides a shocking answer to one of civilization’s oldest questions. Without force—indeed, with just math—we can engineer cooperation between different parties in a well-defined way.

Classifying the set of viable blockchain solutions is not trivial, but it’s clear based on this definition that any problem uniquely solved by blockchain technology will involve some combination of untrusted parties, multiple jurisdictions, high speed, and low costs.

It should also be clear that some solutions currently marketed as blockchain solutions are not actually blockchain solutions, per se, even though they share some of the underlying math (zero-knowledge proofs, homomorphic computing, lattice cryptography, etc). More precisely, all blockchain tech involves cryptographic tech, but not all cryptographic tech involves blockchain tech.

Myth 2: Tokens Are Currencies

Tokens, such as Bitcoins, can be used as currencies, but fundamentally, tokens are not currencies, but capabilities.

Possession of a token gives you the capability to avail yourself of the well-defined, agreed upon services provided by a blockchain technology. Incidentally, people may be willing to give you some form of currency in order to acquire a token, if they want those services, or if they are speculating on the value of the tokens.

For pure cryptocurrencies such as Bitcoin, the services revolve around providing a global, distributed ledger, which blurs the line between capability and currency. But the world doesn’t need more than a few pure cryptocurrencies, so the majority of successful blockchain applications will not be pure cryptocurrencies.

A better analogy for tokens is corporate stock: just like stock gives you capabilities to avail yourself of the services provided by the corporation to stockholders (such as the right to vote, right to dividends, etc.), tokens let you avail yourself of the services provided by the blockchain platform.

Stock usually has real value, because you can often exchange it for money, but it’s less a currency than it is a set of rights to do something.

Myth 3: Blockchain Isn’t Scalable

Today’s blockchains are generally not scalable. Bitcoin can handle a few transactions per second. Ethereum can handle about five times that amount. To give you a sense for how terrible this performance is, Visa handles 65,000 transactions per second!

Now, since tokens are not currency, blockchain platforms don’t necessarily need to scale to the level of payment systems. That said, it’s clear numerous applications require performance orders of magnitude greater than currently offered, as well as far cheaper storage and compute than currently possible on any existing blockchain.

Nonetheless, there are no theoretical reasons why even public blockchain technology cannot scale. Ongoing work in proof-of-stake consensus algorithms, sharding, efficient routing, reputation networks and heterogeneous (specialized) networks, all provide a clear (if difficult) path toward building highly scalable blockchain platforms.

Blockchain platforms can be thought of as highly-constrained, specialized databases. First generation systems are architecturally similar to flat file managers (how data was stored prior to databases). Later systems will be more architecturally similar to highly-scalable databases like Google Spanner.

Myth 4: Blockchain Platforms Will Replicate Tech Stacks

Many people who are sold on the promise of blockchain technology imagine that the distributed applications of tomorrow will be built on blockchain platforms that replicate the form and function of today’s tech stacks.

Just like today’s applications are built on file systems, databases, message queues, compute nodes, and so forth, many imagine that distributed applications will be built on combinations of services like Filecoin (for storage), IPDB (for database), Ethereum (for logic), and so forth.

Billions have been invested in this vision of the future, which is so utterly and catastrophically wrong, entire blockchain platforms will collapse in a heap of rubble. Or at least be reduced to ghosts of their former selves.

The reason is simple: any distributed application that has significant market value can easily attract enough participation to provide all of the services it needs for itself, on its own blockchain, without having to pass along numerous third-party costs to end-users.

The stack for new distributed applications is not going to be a bunch of blockchain platforms, but rather open source software. Developers will assemble their applications from open source components that provide different distributed services, such as queuing and storage.

The only distributed applications that will be built in Frankenstein fashion are those that cannot easily attract enough outside compute and storage—i.e. the set of unsuccessful distributed applications.

Myth 5: Blockchains Are Anti-Government

Lots of early vocal blockchain proponents were outspoken libertarians, or even anarchists, which led many to believe that blockchains are inherently anti-government.

Nothing could be further from the truth.

As tokens are not currency, but assets, proceeds from the sale of such assets can be taxed in the same manner as other assets. In addition, if blockchain technology is eventually regulated differently, the capability to provide verifiable audits on blockchain transactions could ensure perfect compliance with regulation, which is a standard no other industry can match.

Blockchain also provides the technology necessary to power many functions of government in a way that can be audited, and yet preserves a tunable degree of confidentiality in transactions. For example, a government powered by blockchain technology could issue currency and tax participants in an extraordinarily efficient, software-defined way, which leaves no room for tax evasion, has none of the overhead of the existing system, and gives citizens trust through tunable transparency.

Blockchains are not anti-government, they are just a new technological tool, one that actually has the potential to radically improve the efficiency and transparency of government, and usher in an era of software-defined (and therefore, software-enforced) regulation.

Myth 6: Blockchain Is An Append-Only Chain of Blocks

Technically speaking, the word blockchain stems from Bitcoin’s append-only chain of blocks, a data structure produced as part of the “mining” process that confirms transactions.

These days, blockchain technology has evolved past linear append-only chains. Sharded systems utilize Directed Acyclic Graphs (DAGs) of blocks, not linear chains, and there is no specific requirement for ever-growing, append-only chains (we’ll see other types of chains in the future that preserve the ability to audit but discard some historical information).

While some developers may not like the imprecision or terminology drift, blockchain now refers generically to a space of solutions, not any specific implementation techniques or data structures.

Myth 7: Blockchains Should Be Implemented in Go or C/C++

Given the high stakes involved, blockchain platforms should be written in languages that permit as much static verification as possible, and which allow straightforward and easily verifiable translation of math and formal protocol semantics into executable code.

Languages with strong, static type systems, which possess semantics amenable to formal verification, and which support functional programming (a mathematical style of developing software), are an excellent fit for implementation of blockchain technology.

Haskell in particular is proving to be a robust choice for implementing new blockchain technology, and other functional programming languages show promise as well (OCaml, Scala).

(Sorry, I couldn’t resist!)

Myth 8: Blockchain Is a Bubble

No one can perfectly predict the future of Bitcoin, Ethereum, and other major players in the blockchain space—precisely because the success of those platforms depends on people. Developers have to write code, pull requests must be accepted or rejected, and miners must adopt (or not adopt) upgrades.

While it’s impossible to perfectly predict what will become of the first-movers in the blockchain space, I feel confident in saying that the market cap of all blockchain technologies today is completely insignificant compared to what it will ultimately become.

In other words, while individual cryptocurrencies may (or may not) be in a bubble, the overall blockchain market is in its early days. All the real growth lies in the future, not the past.

Blockchain is a technology that will change the world forever.

The Future

In this post, I’ve outlined what I see as the major myths around blockchain technology that impede communication and funnel investment toward dead-end solutions.

What I haven’t done is talk about what blockchain technology is good for.

What are the killer applications for this type of technology, how will they emerge, and how will they be structured? What are the innovations and players we should be paying attention to, and the ones that are safe to ignore? How can blockchain technology provide a radically new way of monetizing some types of open source projects?

All these are excellent topics…for different posts. Let me know what you’re interested in hearing about in the comments below!

John A De Goes bio photo Twitter LinkedIn Github Everything You Know About the Blockchain Is Wrong was published on February 13, 2018 by John A De Goes.

2018/02/14
Infoman

We looked at how a thousand college students performed in technical interviews

37 leeny 9 hrs 24

http://blog.interviewing.io/we-looked-at-how-a-thousand-college-students-performed-in-technical-interviews-to-see-if-where-they-went-to-school-mattered-it-didnt/

news.ycombinator.com/item?id=16369280

interviewing.io is a platform where engineers practice technical interviewing anonymously. If things go well, they can unlock the ability to participate in real, still anonymous, interviews with top companies like Twitch, Lyft and more. Earlier this year, we launched an offering specifically for university students, with the intent of helping level the playing field right at the start of people’s careers. The sad truth is that with the state of college recruiting today, if you don’t attend one of very few top schools, your chances of interacting with companies on campus are slim. It’s not fair, and it sucks, but university recruiting is still dominated by career fairs. Companies pragmatically choose to visit the same few schools every year, and despite the career fair being one of the most antiquated, biased forms of recruiting that there is, the format persists, likely due to the fact that there doesn’t seem to be a better way to quickly connect with students at scale. So, despite the increasingly loud conversation about diversity, campus recruiting marches on, and companies keep doing the same thing expecting different results.

In a previous blog post, we explained why companies should stop courting students from the same five schools. Regardless of your opinion on how important that idea is (for altruistic reasons, perhaps), you may have been left skeptical about the value and practicality of broadening the college recruiting effort, and you probably concede that it’s rational to visit top schools, given limited resources — while society is often willing to agree that there are perfectly qualified students coming out of non-top colleges, they maintain that they’re relatively rare. We here to show you, with some nifty data from our university platform, that this not true.

To be fair, this isn’t the first time we’ve looked at whether where you went to school matters. In a previous post, we found that taking Udacity and Coursera programming classes mattered way more than where you went to school. And way back when, one of our founders figured out that where you went to school didn’t matter at all but that the number of typos and grammatical errors on your resume did. So, what’s different this time? The big, exciting thing is that these prior analyses were focused mostly on engineers who had been working for at least a few years already, making it possible to argue that a few years of work experience smoothes out any performance disparity that comes from having attended (or not attended a top school). In fact, the good people at Google found that while GPA didn’t really matter after a few years of work, it did matter for college students. So, we wanted to face this question head-on and look specifically at college juniors and seniors while they’re still in school. Even more pragmatically, we wanted to see if companies limiting their hiring efforts to just top schools means they’re going to get a higher caliber of candidate.

Before delving into the numbers, here’s a quick rundown of how our university platform works and the data we collect.

The setup

For students who want to practice on interviewing.io, the first step is a brief (~15-minute) coding assessment on Qualified to test basic programming competency. Students who pass this assessment, i.e. those who are ready to code while another human being breathes down their neck, get to start booking practice interviews.

When an interviewer and an interviewee match on our platform, they meet in a collaborative coding environment with voice, text chat, and a whiteboard and jump right into a technical question. Interview questions on the platform tend to fall into the category of what you’d encounter at a phone screen for a back-end software engineering role, and interviewers typically come from top companies like Google, Facebook, Dropbox, Airbnb, and more.

After every interview, interviewers rate interviewees on a few different dimensions, including technical ability. Technical ability gets rated on a scale of 1 to 4, where 1 is “poor” and 4 is “amazing!”. On our platform, a score of 3 or above has generally meant that the person was good enough to move forward. You can see what our feedback form looks like below:

new_interviewer_feedback_circled

On our platform, we’re fortunate to have thousands of students from all over the U.S., spanning over 200 universities. We thought this presented a unique opportunity to look at the relationship between school tier and interview performance for both juniors (interns) and seniors (new grads). To study this relationship, we first split schools into the following four tiers, based on rankings from U.S. News & World Report:

“Elite” schools (e.g. MIT, Stanford, Carnegie Mellon, UC-Berkeley) Top 15 schools (not including top tier, e.g. University of Wisconsin, Cornell, Columbia) Top 50 schools (not including top 15, e.g. Ohio State University, NYU, Arizona State University) The rest (e.g. Michigan State, Vanderbilt University, Northeastern University, UC-Santa Barbara) Then, we ran some statistical significance testing on interview scores vs. school tier to see if school tier mattered, for both interns (college juniors) and new grads (college seniors), comprising a set of roughly 1000 students.

Does school have anything to do with interview performance?

In the graphs below, you can see technical score distributions for interviews with students in each of the four school tiers (see legend). As you recall from above, each interview is scored on a scale of 1 to 4, where 1 is the worst and 4 is the best.

First, the college juniors…

And then, the seniors…

What’s pretty startling is that the shape of these distributions, for both juniors and seniors, is remarkably similar. Indeed, statistical significance testing revealed no difference between students of any tier when it came to interview performance.1 What this means is that top-tier students are achieving the same results as those in no-name schools. So the question becomes: if the students are comparable in skill, why are companies spending egregious amounts of money attracting only a subset of them?

Okay, so what are companies missing?

Besides missing out on great, cheaper-to-acquire future employees, companies are missing out on an opportunity to save time and money. Right now a ridiculous amount of money is being spent on university recruiting. We’ve previously cited the $18k price tag just for entry to the MIT career fair. In a study done by Lauren Rivera through the Harvard Business Review, she reveals that one firm budgeted nearly $1m just for social recruiting events on a single campus.

The higher price tag of these events also means it makes even less sense for smaller companies or startups to try and compete with high-profile, high-profit tech giants. Most of the top schools that are being heavily pursued already have enough recruiters vying for their students. Unwittingly, this pursuit seems to run contrary to most companies desires for high diversity and long-term sustainable growth.

Even when companies do believe talent is evenly distributed across school tiers, there are still reasons for why companies might recruit at top schools. There are other factors that help elevate certain schools in a recruiter’s mind. There are long-standing company-school relationships (for example, the number of alumni who work at the company currently). There are signaling effects too — companies get Silicon Valley bonus points by saying their eng team is comprised of a bunch of ex-Stanford, ex-MIT, ex- etc. etc. students.

So what can companies do?

As such, companies may never stop recruiting at top-tier schools entirely, but they ought to at least include schools outside of that very small circle in the search for future employees. The end result of the data is the same: for good engineers, school means a lot less than we think. The time and money that companies put in to compete for candidates within the same select few schools would be better spent creating opportunities that include everyone, as well as developing tools to vet students more fairly and efficiently.

As you saw above, we used a 15-minute coding assessment to cull our inbound student flow, and just a short challenge leveled the playing field between students from all walks of life. At the very least, we’d recommend employers do the same thing in their process. But, of course, we’d be remiss if we didn’t suggest one other thing.

At interviewing.io, we’ve proudly built a platform that grants the best-performing students access to top employers, no matter where they went to school or where they come from. Our university program, in particular, allows us to grant companies the privilege to reach an exponentially larger pool of students, for the same cost of attending one or two career fairs at top target schools. Want diverse, top talent without the chase? Sign up to be an employer on our university platform!

2018/02/14
Infoman

When Apple released ARKit, designing for augmented reality wasn’t defined as a field. But during autumn of 2017 in App Store appeared a lot of AR apps. Almost all of them used ARKit. In the beginning, user flow of augmented reality applications wasn’t clear. I wanted to define what interaction patterns are the most popular in ARKit apps. So, I made an extensive analysis of onboarding process for almost all apps available in App Store. Also, I have made UX reviews for free just to get experience and more insights.

I keep noticing that for the first time users was tough to understand how AR works. Also, UI wasn’t clear even for experienced in AR users. So learning curve for new users became even longer.

It looks like first demos build mostly by Unity and iOS developers. Obviously, they didn’t have the time or any other resources to hire experienced UX designer. Those times there wasn’t settled interactions patterns that could be followed.

Thus in my article “Onboarding in augmented reality mobile application” I tried to cover the essential steps and clarify on what to focus and what to avoid. I hope that it made at least single app more user-friendly.

After some time Apple and Google released own guidelines for AR apps. Although they are beneficial, they still left blind spots. I do recommend to have a look if you didn’t have a chance to learn them.

While the time I collected a lot of screenshots and started noticing some pattern in building user flows and using UI components in AR apps. So I decided to create UI kit that could be sophisticated enough for designers, but also can be useful for developers that can find suitable UI solutions to copy.

For easy searching I sorted all screens in next categories:

It’s first and the most confusing moment for first-time users. An app should gently (or not) let a user know that it needs some info about the environment.

When application scanned area and detected planes, a user can start placing 3d content. There are different interaction patterns, that can be suitable for different situations.

This step is about playing with augmented objects itself. In flat UI there are a lot of stable well-know patters, as swipe from the top to update page, and so on. In AR it’s like the wild west. Apple and Google tried to define some situations. But sometimes case require unique interactions. It’s critical to introduce them gently. Depending on the importance it can be small tip net to the button or almost full page popup.

Maybe the most well known AR app is Pokémon GO, what is location based. I hope that in future we will get much more interesting cases of using location-based augmented reality

For some applications, AR isn’t the main feature. Show object in AR is the only option. Apple suggested the icon for consistency. But I haven’t noticed it in any application that I tested. Also, lists and galleries are the most obvious ways to find a model.

All his time user spends in camera view. There can be various UI components that help interact with AR world.

To quickly explain some gestures and concepts to a user I created the collection of icons:

I hope that this collection will help to create more user-fieldy augmented reality mobile applications.

2018/02/12
Infoman

Place Kit – The future of augmented reality mobile apps

9 VovaKurbatov 4 hrs 1

https://medium.com/inborn-experience/place-kit-the-future-of-augmented-reality-mobile-apps-25a20907316b

news.ycombinator.com/item?id=16356659 When Apple released ARKit, designing for augmented reality wasn’t defined as a field. But during autumn of 2017 in App Store appeared a lot of AR apps. Almost all of them used ARKit.

In the beginning, user flow of augmented reality applications wasn’t clear. I wanted to define what interaction patterns are the most popular in ARKit apps. So, I made an extensive analysis of onboarding process for almost all apps available in App Store. Also, I have made UX reviews for free just to get experience and more insights.

I keep noticing that for the first time users was tough to understand how AR works. Also, UI wasn’t clear even for experienced in AR users. So learning curve for new users became even longer.

It looks like first demos build mostly by Unity and iOS developers. Obviously, they didn’t have the time or any other resources to hire experienced UX designer. Those times there wasn’t settled interactions patterns that could be followed.

Thus in my article “Onboarding in augmented reality mobile application” I tried to cover the essential steps and clarify on what to focus and what to avoid. I hope that it made at least single app more user-friendly.

After some time Apple and Google released own guidelines for AR apps. Although they are beneficial, they still left blind spots. I do recommend to have a look if you didn’t have a chance to learn them.

While the time I collected a lot of screenshots and started noticing some pattern in building user flows and using UI components in AR apps. So I decided to create UI kit that could be sophisticated enough for designers, but also can be useful for developers that can find suitable UI solutions to copy.

For easy searching I sorted all screens in next categories:

It’s first and the most confusing moment for first-time users. An app should gently (or not) let a user know that it needs some info about the environment.

When application scanned area and detected planes, a user can start placing 3d content. There are different interaction patterns, that can be suitable for different situations.

This step is about playing with augmented objects itself. In flat UI there are a lot of stable well-know patters, as swipe from the top to update page, and so on. In AR it’s like the wild west. Apple and Google tried to define some situations. But sometimes case require unique interactions. It’s critical to introduce them gently. Depending on the importance it can be small tip net to the button or almost full page popup.

Maybe the most well known AR app is Pokémon GO, what is location based. I hope that in future we will get much more interesting cases of using location-based augmented reality

For some applications, AR isn’t the main feature. Show object in AR is the only option. Apple suggested the icon for consistency. But I haven’t noticed it in any application that I tested. Also, lists and galleries are the most obvious ways to find a model.

All his time user spends in camera view. There can be various UI components that help interact with AR world.

To quickly explain some gestures and concepts to a user I created the collection of icons:

I hope that this collection will help to create more user-fieldy augmented reality mobile applications.

Check full preview on Behance

2018/02/12
Infoman

Test

2018/02/12
Infoman

Html5 recording plus flask server

https://henri.io/posts/streaming-microphone-input-with-flask.html

2018/02/10
Infoman

Online restaurant gets money from fund managed by SingPost’s ex-chairman

Photo credit: Grain

Despite going through a crisis last year, Grain, a Singapore-based startup that makes fresh meals in its own kitchen and sells them online, has raised an undisclosed sum of money from investors that include Majuven, a venture capital fund founded by Lee Hsien Yang, the brother of Singapore’s prime minister.

Lim Ho Kee, the former chairman of Singapore’s national postal company SingPost, is Majuven’s managing partner.

Grain disclosed these figures related to its 2017 performance:

330 percent growth in sales Gross profit grew more than 10 percent 60 percent repeat customers Here are some numbers obtained from its income statement as well as from a source familiar with the matter. The person did not disclose earnings for the latest financial year that ended.

Grain has come a long way. It almost ran out of money at one point and was reprimanded by Singapore’s health authority for selling food that caused diarrhea. It later apologized and said it identified lapses in the kitchen that were fixed. The company then hired a new executive chef who ran the kitchen at Google and food caterer Neo Group.

Grain only serves customers in Singapore. New investors in this round include food distributor FoodXervices and restaurant group First Gourmet. Existing backers NSI Ventures, Thai Express founder Ivan Lee, and 500 Startups also participated.

The investment will be used to grow Grain’s country-wide distribution network before its series B fundraising. Besides serving individual meal boxes to consumers, Grain also serves businesses like Yahoo, DBS, TripAdvisor, Oldendorff, and Danone.

The company previously raised a series A round worth US$1.7 million, a bridge round of roughly the same amount, as well as some venture debt from DBS Bank.

Can Grain survive the platforms? Grain’s business model of owning the entire process – from food preparation to delivery – is a brave but difficult choice. Small restaurants often rely on marketplaces like Uber Eats and Deliveroo to do deliveries for them. Only big restaurant chains can afford to send out their own riders.

On the other hand, because Grain only delivers food, it does not need to deal with the complexity of renting space and running physical eateries. And compared to a marketplace model, it does not need to reach as big a scale to become profitable. (That said, bringing costs down is still critical if it wants its investors to make good returns.)

Yet these marketplaces are starting to mount a challenge to Grain’s business model. Deliveroo, for example, has rented its own kitchens where restaurants on its app can cook dishes and deliver them to customers far away from their physical locations.

UberEats is also pushing the virtual restaurant trend by partnering with establishments to create dishes that only exist in its app. These meals are offered by dining places that do not exist in an actual locale.

These moves from digital to physical are inevitable. Once platforms grow bigger, they will own more and more of the supply chain in order to improve their bottom line.

Grain can thrive alongside these platforms, but it needs to avoid becoming just another forgettable restaurant. And that starts with serving delicious food at reasonable prices. In other words, it needs to become an online-only, healthier version of McDonald’s.

Improving its kitchen processes, getting strategic investors in the food industry, and hiring an experienced executive chef are all steps in the right direction.

Grain’s financial numbers were converted from Singapore dollars. US$1 = S$1.32.

Update (Feb. 8, 6:05 pm): Added more details and analysis.

Grain Grain is Restaurant 2.0. Our mission is to make great food sustainable and accessible for everyone. We are an online restaurant that leverages data and a tech-enabled distribution network to serve food to customers.

LOCATIONSingaporeFOUNDED2014EMPLOYEES - WEBSITEwww.grain.com.sgLATEST FUNDINGUnspecified StageHIRING6 positions Editing by Eileen C. Ang

(And yes, we’re serious about ethics and transparency. More information here.)

TIA Writer Terence Lee I let my work do the talking. Got a news tip? Email me at terence@techinasia.com. VIEW ALL COMMENTS CONFERENCE

2018/02/08
Infoman

How Go-Jek compares to other ‘super apps’ like WeChat

Photo credit: Go-Jek

Asia’s increasingly crowded tech space has resulted in super apps–applications that consolidate versatile, multi-functional features such as chat, social media, mobile payments, games, and much more.

Some of the most notable examples include WeChat, Facebook Messenger, and Line. One rising star in the super app space is Indonesia’s transportation service Go-Jek.

Why Go-Jek succeeds in Indonesia Go-Jek was originally a service that aimed to provide inexpensive motorcycle rides to users. Over time, the company has evolved, becoming able to transport people and goods and even provide services. Indonesians are mobile-first. The vast majority of the country’s internet traffic originates from mobile devices, and Go-Jek is thriving within this framework.

What makes it different from other super apps? Go-Jek’s path to become a super app is markedly different than that of its peers. The three key distinguishing factors are its transportation-based roots, its non-emphasis on social media, and its lack of a powerful parent company.

For one, Go-Jek’s path to success hinged on a successful transportation service. The company filled a unique need for a country dependent on ojeks, something that would be much more difficult in non-urban American and Chinese environments.

It also doesn’t emphasize social media services. WeChat, Messenger, and Line all focused on establishing such capabilities, capitalizing on a distinct need for a well-supported and widely-accepted social media platform in their respective countries. With this path, Go-Jek has arguably created a larger base of customers, including the 35-to-64 age segment who own smartphones but do not necessarily use social media.

Lastly, the company doesn’t have a parent company with multiple investments in other sectors. This results in a dramatic difference in global reach and expansion possibilities.

In China, WeChat backer Tencent is a prolific investor in Chinese-related internet services, allowing them to reach both domestic and overseas Chinese users. Meanwhile, Line has a wide user base outside of its native Japan.

On the other hand, Go-Jek’s current user base is exclusively Indonesian. This means they may lack international experience and capabilities that some of its peers have.

Similarities with its peers In spite of these differences, Go-Jek does share a variety of similarities with its peers: the initial stages of super app growth, the expansion of the super app into various markets, and the acquisition of seemingly “unrelated” companies.

The first similarity is Go-Jek’s success in a singular industry. While the industries differ between super apps, each of them focused on a single product and built it into lateral and vertical ecosystems. By pursuing this strategy, WeChat, Messenger, Line, and Go-Jek have been able to grow exponentially.

The similarities extend to the super apps’ expansions into vertical and horizontal markets. WeChat decided to expand into social media through “Moments,” mobile payment through WePay, and search engines. Similarly, Messenger introduced “Stories,” mobile payment services, games, etc. Line’s expansion is similar with Line Taxi, mobile internet games, and even an original animated series.

Similarly, Go-Jek has expanded into digital wallets and food delivery, marking over 15 million weekly active users.

Lastly, the super apps I mention here expand through acquisitions in both vertical and horizontal markets. Tencent has established a strong acquisitions presence—acquiring companies such as Riot Games while also investing in startups like Vipshop. Backing its Messenger app, Facebook has also acquired companies like Instagram and WhatsApp. Line Corporation, on the other hand, acquired a variety of companies including startups WebPay, TemanJalan, and FIVE.

Just two months ago, Go-Jek announced the addition of three Indonesian fintech companies into its portfolio: Kartuku, Midtrans, and Mapan. These companies—which focus on offline payments, online payments, and community-based lending respectively—will serve to reinforce Go-Jek’s status as a mobile payment provider. Go-Jek has also acquired ticketing and live event management company Loket, Indian healthcare startup Pianta, and mobile-app development firm Leftshift, further expanding itself into other verticals.

Go-Jek’s backers and future challenges Since its founding in 2010, Go-Jek has received a variety of high-profile backers. The beginning of 2018 also brought the company closer to its next funding round, with commitments from Google, Temasek, and Meituan-Dianping.

It will also be interesting to see Go-Jek develop itself in the mobile payments space. While the company has immense potential in this market, it will face institutional challenges in the country, where many are still wary of such payment systems and cash-on-delivery remains king.

Go-Jek is also facing tough competition from Singapore-based Grab, another prominent force in the Southeast Asian ride-sharing market. Go-Jek must determine its path and slowly spread itself into separate markets if it aims to retain its huge market share.

Ultimately, Go-Jek’s climb to super app status is crucial for the Indonesian startup scene, as it would blaze a path for other Indonesian startups to thrive in the regional and global markets.

Editing by Jaclyn Teng

(And yes, we’re serious about ethics and transparency. More information here.)

2018/02/08
Infoman

近日美股（以至環球股市）出現近乎閃崩式的暴跌，市場對促成是次急插的因由眾說紛紜。姑勿論背後實質理由，投資者目前最關注的問題是，究竟經過近日的暴跌後，美股及港股的升浪「升完未」？後市還會有牛三最後升浪嗎？

環球股市踏入2月後表現可謂驚濤駭浪，美國道瓊斯工業平均指數日波幅動輒便逾千點。在短短6個交易日內，道指已由26616點的歷史高位，一度急插近2700點（下挫逾一成）。某程度反映市場恐懼情緒的VIX指數，亦由年初9.15%低位，顯著攀升至逾三成高位，創2015年以來的高位。

美股閃崩因由眾說紛紜

市場對促成是次美股閃崩的因由眾說紛紜，惟大致可歸納為：（1）通脹預期升溫，並且擔憂聯儲局加快加息的步伐；（2）債息升勢過急；（3）程式沽盤交易湧現；（4）美股早前已嚴重超買，借勢出現調整。

無疑，市場早前確實預期聯儲局加息步伐相對緩慢（兩次起、三次止），不過，也非完全對通脹升溫沒有任何警覺。事實上，從可反映通脹預期的打和利率（breakeven interest rate）可見，不同檔期的打和利率早在去年底已出現不同程度的抽升【圖1】，反映市場對通脹升溫非完全沒有心理準備。當然，上周五就業報告（尤其時薪增長）強勁來得較突然，或令市場有所錯愕。

話雖如此，這也未必可以確認通脹會出現飆升，甚至令聯儲局「短期內」有加快上調息率的壓力。事實上，聖路易斯聯儲銀行行長布拉德（James Bullard）近日已指出，「部分人薪酬升幅大，並不一定會導致更高的通脹」，暗示聯儲局未必因為上周的就業數據，而短期加快加息步伐。說到底，始終美股近年來連綿的升浪，一直都沒有出現過像樣的調整，早前已出現極度的超買，例如標普500指數周線RSI早前的讀數高達90.5超買位置，創1927年有記錄以來的高位；故近日藉通脹升溫等消息，加上程式交易沽盤推波助瀾下，令美股以至環球股市終出現遲來的調整。

加息難促成股市轉向

目前美股（及環球股市）的震盪，是否已經完結呢？相信還有待確認（惟筆者估計尚未完成）；反而，更重要的是，究竟近日股市的暴瀉，是否意味整個牛市升浪已結束呢？筆者相信是「未結束」，並且保持恒指今年保守見36000點，最牛看39000點的觀點。主要理據如下：

先要明白的是，加息往往都不是促成股市轉向的因素。根據多年來聯儲局貨幣政策周期與股市表現的關係，可清晰看到加息周期進行中，股市的表現往往處於反覆向上；反而，聯儲局減息期間，美股走勢相對較差。這點也許不難理解，經濟差，即反映企業盈利表現轉差，聯儲局才有需要減息，股市往往亦會在這環境下表現較弱，反之亦然【圖2】。換言之，僅從加息因素看，不應（也不會）對股市造成太大及持久的壓力，尤其目前的利率水平仍屬偏低。故此，縱使聯儲局如本欄預期，今年加息幅度達四次或以上，相信不會對經濟、進而股市造成太大的壓力，遑論對股市走勢帶來轉向的影響。

其次，美國國債牛市已確認結束。從目前十年長債債息的水平來看，已向上突破運行逾30年的下降通道頂線【圖3】。而且今年聯儲局還有繼續加息的空間，料只會進一步推升債息水平（即令債價繼續受壓）。換言之，美國國債牛市確認完結應該沒有太大懸念。

「債股輪動」屢有先例

若然美債牛市結束，重要思考的問題是，錢會去哪兒？正如去年底本欄的分析，2018年投資市場主軸（主題）之一，便是「債股輪動」，即債市沽壓釋出資金流入股市，帶動股市繼續造好。

先要了解的是，自金融海嘯後，聯儲局推出浪接浪，規模一個比一個龐大的量化寬鬆政策（QE），進一步延續債市牛市，期間亦引導數以萬億美元計的資金湧入債市。不過，目前美國國債牛市完結，預料龐大資金會「逐步」流出債市，股市相信是其中之一的主要目的地，尤其目前企業盈利前景尚算不俗（註：因為今年環球通脹亦開始升溫，相信資金流入其他國家債券及貨幣市場基金的量會相對較少）。

事實上，類似「債市不濟，股市不俗」的情況不時出現。自1930年至今，美國10年期長債債價出現按年跌幅，曾合共發生過16次。而當年美股（標普500指數）的表現，16年中便有13年錄得升幅，即上升比例高達八成以上【圖4】。例外的年份只有1931年（出現大蕭條初期）、1941年（二次大戰期間）和1969年。

當然，「債股輪動」令股市受惠的一個重要假設，是資金流出債券市場的速度不要太快，或債息回升速度不宜過急。要此情況出現，其中兩個重要條件是，（1）通脹升溫速度不快（或通脹沒有急速惡化）和（2）市場對美國國債信心尚在，沒有出現恐慌拋售情況（即美債泡沫未出現爆破）。

無疑，隨着經濟擴張踏入第三年，預料今年通脹確有所回升，惟相信上升速度不致失控地惡化。此外，從近日金融市場震盪，還可以發現資金有流入債市避險的蹤影，顯示縱使債市牛市結束，市場暫未致對美國國庫券完全失去信心，甚至出現恐慌性拋售。故此，相信這有利「股債輪動」，進一步推升股市。

總而言之，雖然近日市場波動明顯擴大，短期甚至還有進一步調整的空間，惟較中線角度（未來6個月）來看，今年通脹未致失控上升，市場亦未因為美債牛市結束而急速拋售美債，故此相信在「債股輪動」的資金流向驅使下，今年美股和港股（及新興市場股市）牛市還有一段最後升浪要走。筆者對港股和美股上半年走勢依然審慎樂觀；君不見近日雖然市場異常波動，但資金依然保持流入新興市場嗎？不過，要注意今年股市走勢模式已再不是慢牛，而是瘋牛，即大上大落。故入市時要注意風險，量力而為。

信報投資研究部

2018/02/08
Infoman

from ABBYY import CloudOCR

ocr_engine = CloudOCR(application_id='uniteach', password='+OPMYtXUt6Bi5YAyuBvPDsW4')

pdf = open('https://unitest.click/static/uploadfiles/img-4555d3b79a424b23838822c1850e6488.jpg', 'rb')

file = {jpg.name: jpg}

result = ocr_engine.process_and_download(file, exportFormat='txt,pdfTextAndImages', language='English')

2018/02/08
Infoman

Growing from 0 to 4M users on our fashion app with vertical machine learning

168 aldamiz 5 hrs 26

https://medium.com/@aldamiz/f8b7fc0a89d7

news.ycombinator.com/item?id=16325945

Chicisimo for iOS. Download iOS or Android Three years ago we launched Chicisimo, our goal was to offer automated outfit advice. Today, with over 4 million women on the app, we want to share how our data and machine learning approach helped us grow. It’s been chaotic but it is now under control.

If we wanted to build a human-level tool to offer automated outfit advice, we needed to understand people’s fashion taste. A friend can give us outfit advice because after seeing what we normally wear, she’s learnt our style. How could we build a system that learns fashion taste?

We had previous experience with taste-based projects and a background in machine learning applied to music and other sectors. We saw how a collaborative filtering tool transformed the music industry from blindness to totally understanding people (check out the Audioscrobbler story). It also made life better for those who love music, and created several unicorns along the way.

With this background, we built the following thesis: online fashion will be transformed by a tool that understands taste. Because if you understand taste, you can delight people with relevant content and a meaningful experience. We also thought that “outfits” were the asset that would allow taste to be understood, to learn what people wear or have in their closet, and what style each of us like.

Online fashion will be transformed by a tool that understands taste. Because if you understand taste, you can delight people. “Outfits” are the asset that allows taste to be understood We decided we were going to build that tool to understand taste. We focused on developing the correct dataset, and built two assets: our mobile app and our data platform.

From previous experience building mobile products, even in Symbian back then, we knew it was easy to bring people to an app but difficult to retain them. So we focused on small iterations to learn as fast as possible.

We launched an extremely early alpha of Chicisimo with one key functionality. We launched under another name and in another country. You couldn’t even upload photos… but it allowed us to iterate with real data and get a lot of qualitative input. At some point, we launched the real Chicisimo, and removed this alpha from the App Store.

We spent a long time trying to understand what our true levers of retention were, and what algorithms we needed in order to match content and people.

Three things helped with retention:

Releasing changes into the app. (a) identify retention levers using behavioral cohorts (we use Mixpanel for this). We run cohorts not only over the actions that people performed, but also over the value they received. This was hard to conceptualize for an app such as Chicisimo. We thought in terms of what specific and measurable value people received, measured it, and run cohorts over those events, and then we were able to iterate over value received, not only over actions people performed. We also defined and removed anti-levers (all those noisy things that distract from the main value) and got all the relevant metrics for different time periods: first session, first day, first week, etc. These super specific metrics allowed us to iterate (Nir Eyal’s book Hooked: How to Build Habit-Forming Products discusses a framework to create habits that helped us build our model);

(b) re-think the onboarding process, once we knew the levers of retention. We define it as the process by which new signups find the value of the app as soon as possible, and before we lose them. We clearly articulated to ourselves what needed to happen (what and when). It went something like this: If people don’t do [action] during their first 7 minutes in their first session, they will not come back. So we need to change the experience to make that happen. We also run tons of user-tests with different types of people, and observed how they perceived (or mostly didn’t) the retention lever;

(c) define how we learn. The data approach described above is key, but there is much more that data when building a product people love. In our case, first of all, we think that the what-to-wear problem is a very important one to solve, and we truly respect it. We obsess over understanding the problem, and over understanding how our solution is helping, or not. It’s our way of showing respect.

This leads me to one of the most surprising aspects IMO of building a product: the fact that, regularly, we access new corpuses of knowledge that we did not have before, which help us improve the product significantly. When we’ve obtained these game-changing learnings, it’s always been by focusing on two aspects: how people relate to the problem, and how people relate to the product (the red arrows in the image below). There are a million subtleties that happen in these two relations, and we are building Chicisimo by trying to understand them. Now, we know that at any point there is something important that we don’t know and therefore the question always is: how can we learn… sooner?

This model helps us think. I understand that there are other models and this one might be controversial. Talking with one of my colleagues, she once told me, “this is not about data, this is about people”. And the truth is, from day one we’ve learnt significantly by having conversations with women about how they relate with the problem, and with solutions. We use several mechanisms: having face to face conversations, reading the emails we get from women without predefined questions, or asking for feedback around specific topics (we now use Typeform and it’s a great tool for product insight). And then we talk among ourselves and try to articulate the learnings. We also seek external references: we talk with other product people, we play with inspiring apps, and we re-read articles that help us think. This process is what allows us to learn, and then build product and develop technology.

At some point, we were lucky to get noticed by the App Store team, and we’ve been featured as App of the Day throughout the world (view Apple’s description of Chicisimo, here). On December 31st, Chicisimo was featured in a summary of apps the App Store team did, we are the pink “C.” in the left image below 😀.

The app got viewed by 957,437 uniques thanks to this feature, for a total of 1.3M times. In our case, app features have a 0,5% conversion rate from impression to app install (normally: impression > product page view > install); ASO has a 3% conversion, and referrers 45%.

The app aims at understanding taste so we can do a better job at suggesting outfit ideas. The simple act of delivering the right content at the right time can absolutely wow people, although it is an extremely difficult utility to build.

Chicisimo content is 100% user-generated, and this poses some challenges: the system needs to classify different types of content automatically, build the right incentives, and understand how to match content and needs.

We soon saw that there was a lot of data coming in. After thinking “hey, how cool we are, look at all this data we have”, we realized it was actually a nightmare because, being chaotic, the data wasn’t actionable. This wasn’t cool at all. But then we decided to start giving some structure to parts of the data, and we ended inventing what we called the Social Fashion Graph. The graph is a compact representation of how needs, outfits and people interrelate, a concept that helped us build the data platform. The data platform creates a high-quality dataset linked to a learning and training world, our app, which therefore improves with each new expression of taste.

We thought of outfits as playlists: an outfit is a combination of items that makes sense to consume together. Using collaborative filtering, the relations captured here allow us to offer recommendations in different areas of the app.

Screenshots of one of our patents. Chicisimo’s first source of data relations, and first representation of the Social Fashion Graph. There was still a lot of noise in the data, and one of the hardest things was to understand how people were expressing the same fashion need in different ways, which made matching content and needs even more difficult. Lots of people might need ideas to go to school, and express that specific need in a hundred different ways. How do you capture this diversity, and how do you provide structure to it? We built a system to collect concepts (we call them needs) and captured equivalences among different ways to express the same need. We ended up building a list of the world’s what-to-wear needs, which we call our ontology. This really cleaned up the dataset and helped us understand what we had. This understanding led to better product decisions.

We now understand that an outfit, a need or a person, can have a lot of understandable data attached, if you allow people to express freely (the app) while having the right system behind (the platform). Structuring data gave us control, while encouraging unstructured data gave us knowledge and flexibility.

Chicisimo The end result is our current system. A system that learns the meaning of an outfit, how to respond to a need, or the taste of an individual.

And I wouldn’t even dare saying that this is Day 1 for us.

Screenshot of an internal tool.

Each Need has different data attached to it. It belongs to a certain category; it is correlated to other needs; it is attached to different types of outfits and shoppable products; it plays a certain role in the ontology… The amount of work we have in front of us is immense, but we feel things are now under control. One of the new areas we’ve been working on is adding a fourth element to the Social Fashion Graph: shoppable products. A system to match outfits to products automatically, and to help people decide what to buy next. This is pretty exciting.

Shoppable sweaters currently available via a ShopStyle integration Back when we built recommender systems for music and other products, it was pretty easy (that’s what we think now, we obviously didn’t think that at the time:). First, it was easy to capture that you liked a given song. Then, it was easy to capture the sequence in which you and others would listen to that song, and therefore you could capture the correlations. With this data, you could do a lot.

However, as we soon found out, fashion has it’s own challenges. There is not an easy way to match an outfit to a shoppable product (think about most garments in your wardrobe, most likely you won’t find a link to view/buy those garments online, something you can do for many other products you have at home). Another challenge: the industry is not capturing how people describe clothes or outfits, so there is a strong disconnect between many ecommerces and its shoppers (we think we’ve solved that problem. Also Similar.ai and Twiggle are working on it). Another challenge: style is complex to capture and classify by a machine.

Now, deep learning brings a new tool to add to other mechanisms, and changes everything. Owning the correct data set allows us to focus on the specific narrow use cases related to outfit recommendations, and to focus on delivering value through the algorithms instead of spending time collecting and cleaning data. 👉 Now comes the fun and rewarding part, so please email us if you want to join the team and help build algorithms that have real impact on people — we are 100% remote, Slack based 👈 -😂😂😉 😉 😉. People’s very personal style can become as actionable as metadata and possibly as transparent as well (?), and I think we can see the path to get there. As we have a consumer product that people already love, we can ship early results of these algorithms partially hidden, and increase their presence as feedback improves results.

What type of looks should we suggest someone? There are more and more researchers working of these areas, you can read Tangseng’s paper on recommending outfits from personal closet or clothing parsing project, or how Edgar Simo-Serra defines similarity between images using user-provided metadata.

Outfits are a key asset in the race to capture the $123 billion US apparel market. Data is also the reason many players are taking outfits to the forefront of technology: outfits are a daily habit, and have proven to be great assets to attract and retain shoppers, and capture their data. Many players are introducing a Shop the Look section with outfits from real people: Amazon, Zalando or Google are a few examples.

Google recently introduced a new feature called Style Ideas showing how a “product can be worn in real life”. Same month Amazon launched it’s Alexa Echo Look to help you with your outfit, and Alibaba’s artificial intelligence personal stylist helped them achieve record sales during Singles Day.

Apparel, footwear and accessories retail e-commerce revenue in the US (source). “Clothing and shoes“ is the largest product category in the US (source). Some people think that fashion data is in the same place as music data was in 2003: ready to play a very relevant role. The good news is: the daily habit of deciding what to wear will not change. The need to buy new clothes won’t disappear, either.

So, what do you think? Where will we be 10 years from now? Will taste data build unique online experiences? What role will outfits play? How will machine learning change fashion ecommerce? Will everything change, 10 years from now?

We are a small team of eight, four on product and four engineers. We believe in focusing on our very specific problem, no one on earth can understand the problem better than us. We also believe on building the complete solution ourselves while doing as few things as possible. We work 100% remote and live in Slack + GitHub. You can learn more about our machine learning approach, here.

If you are a deep learning engineer or a product manager in the fashion space, and want to chat & temporarily access our Social Fashion Graph, please email us describing your work. You can also download our iOS and Android apps, or simply say hi: hi at chicisimo.com.

2018/02/07

«
1
2
…
143
144
145
146
147
148
149
…
183
184
»