AiLab Interview with Jeremy Howard

AiLab interviewed Jeremy Howard, a data scientist, entrepreneur and co-founder of fast.ai. An Australian who has been based in San Francisco (California, USA) for over a decade, we spoke to Jeremy to get his expert insights into Artificial Intelligence and deep learning just before he moved back to Australia.

This interview took place in February 2021 with Dr. John Flackett.

AiLab: Hi Jeremy. Thanks for speaking with me today. Could you tell us a little bit about yourself, especially your research and work history?

JH: Sure, it's a little complicated and strange background. My main interest and the focus of my research and all my work, is making deep learning more accessible. My wife and I started a research, teaching and development lab to do that called fast.ai. We did that because we thought a few years ago that deep learning was a really powerful tool, but that it was about to become a much more powerful tool that could be used for lots of different things.

At that time back in 2013, the only people who could wield this tool were a very small group of basically, white men with technical skills attached to a small number of esteemed universities. We felt this was a shame because we knew a lot of these people, and although they're good, smart folks, we knew they were not familiar with critical issues like water availability, global education access, racial inequality and so on. For example, the kind of problems they were solving with deep learning were, How can I make it more convenient to organize my photo library on my iPhone?  Instead of, How do we avoid mass deaths due to lack of access to water?

So rather than doing the thing which everybody was talking about which was, Oh, let's hire some of these brilliant deep learning geniuses into our company, we thought let's show organisations that they already have brilliant people who are perfectly capable of learning these skills. At that time it wasn't at all clear that this was even possible, but we thought it probably was because it's not magic. So we started with teaching, and that was hard because there wasn't any place to go to figure out what to teach; there's a lot of tricks that were literally secrets at that time. We had to figure a lot out before we could start teaching. There was a lot of research and a lot of this stuff was hard to implement, so we basically started writing software during the research to support the teaching — which is what I’m still doing today!

As to my work history, I've been coding for well over 30 years. One of my favorite things to do is to write code. I spent nearly a decade in business strategy consulting with McKinsey and AT Kearney. I then spent nearly a decade creating and running startups in Australia; including FastMail (an email company) and Optimal Decisions (an insurance price optimization company). The most recent few years I’ve been very focused on machine learning with Kaggle and Enlitic (which was the first company to focus on deep learning in medicine), and then fast.ai along with an appointment as a Distinguished Research Scientist at the University of San Francisco.

AiLab: Wow, that’s quite a resume!

JH: It's a bit all over the place. [Laughs]

AiLab: In democratising deep learning and the reasons for doing it, I’ve heard you talk about the importance of real world problems in education?

JH: We try to give the skills to people who focus on real world problems. At Enlitic I was focusing on specific real world problems in medicine. That was kind of cool, but it was also frustrating because there were so many real world problems I wasn't helping with and there's no way I can do all of them. Hence the idea of putting these tools into the hands of people who are already working on these problems and who already have access to the data.

Most people want to not just learn a skill, but do something useful with it. The normal way that deep learning is taught is very bottom-up. You can spend years learning linear algebra, then years learning calculus, then years learning statistics, and maybe eventually you'll prove some theorem about error bounds on optimisation algorithms, but what you really wanted to do is build something – for example, help a drone find earthquake hit areas. So we think the best way to learn is, as much as possible, by doing. A lot of people have spent years and years thinking that they're learning deep learning and then finding out much later they're hopeless at it. [Laughs]. They can't actually make a predictive model that predicts things accurately or fast enough. Then they realise what you need to learn to do those things is not about proving error bounds of your optimisation algorithm at all. It's about numerical programming tricks, parallelisation of algorithms, understanding how to profile memory, and stuff like that.

AiLab: Have you got any advice for companies that are looking to employ AI tools and techniques?

JH: We do this top-down method of teaching where from lesson one you get a data set that you care about and start building models with it. So people in an organisation should go through this process of trying to actually build some models as they learn. The early attempts are not going to go straight into production, but they are getting some real world feedback on what's working or what's not working. They're writing code, munging data, visualising the actual things they care about, and figuring out where data they need even exists in the organisation. So the way I see it is that organisations should focus on the skills and expertise that are being developed in their people and they should be pragmatic. Which is not to say that you get to skip math entirely. Obviously there's plenty of math, but you deal with that as you come to it. It turns out that in the world of linear algebra, there's like 0.01% you actually have to know to be any good at deep learning; for calculus, maybe even less.

I find organisations have a tendency to want to outsource AI. They'll try to do a contract with a big service provider or they'll try to hire somebody from some university at great expense. It's like outsourcing a core competency to people who have no idea what your company does, why it does it, what the constraints are, what the opportunities are, what the strategy is, or what the data sources are. To me it's weird. For example, if you’re an insurance company it seems it would be insane to consider doing that for claims assessment, but somehow it's OK for your AI strategy.

AiLab: Starting from a problem solving point of view rather than a maths perspective also helps make AI much more accessible?

JH: People think deep learning is like some kind of unicorn rainbow magic that sprinkles down and that there are certain people that have the magic, and when you find them their brains are four times bigger than everybody else and they were born with optimisation algorithms in their heads. [Laughs]. This is part of the problem; when things seem magic they are inaccessible. People believe that they're not smart enough to understand it and then they don't believe they have the capability to even make sensible decisions to judge ideas — when in fact you judge them exactly the same way as any other organisational idea.

So one of the things we designed in fast.ai and the ‘Deep Learning for Coders with fastai and PyTorch’ book (especially the first 3 chapters) is that the content is readable by people that can't actually code and can barely remember high school math. This really helps with the general understanding, so if you do hire a vendor to build some models, you know what they are doing, how might it go wrong and what to tell them if it does go wrong. It’s important to protect your own business and set realistic expectations; these systems are built on your data and so if there’s a scenario you haven't seen before in your business and it’s not in your data, then the system can't accurately predict that particular thing.

When you start a new approach to making automated decisions in your business, start small. It's all common sense stuff, but we try to write it down to make each step pretty clear.

AiLab: I like the way that you take promising work and present it in ways that people can understand.

JH: Often the promising work that we find and evangelise is ridiculously simple. Anytime something looks like it needs a large amount of compute, huge amounts of data or complicated math, we try to find a way to get rid of those bits, because those are all things that make the technology less accessible. For example, Jeff Dean at Google is a huge fan of automated machine learning (AutoML). The idea, as Jeff puts it, is to have thousands of times fewer data scientists and to use thousands of times more compute to have the computers setting the hyperparameters.

But our approach was to spend a few days studying hyperparameters and figuring out how to set them correctly, first time, every time. Now, the idea that setting hyperparameters better is going to require thousands of times fewer data scientists is crazy, because data scientists only spend half of a percent of their time setting hyperparameters. However, it's a good excuse for selling lots of additional compute in the Google data centres. So there are these terrible conflicts of interest of companies like Google wanting to sell us techniques and solutions, which require huge amounts of compute.

There are very few organisations like fast.ai that are explicitly trying to do the exact opposite, which is to find ways to do things with dramatically less compute. It's really important, because otherwise you get the haves and the have-nots; not about expertise but about money. It will always be true that people with money can make more money, but I don't want that to be anymore true then it already is.

AiLab: You mentioned earlier that some people perceive AI as magical, so providing AI education and setting realistic expectations is really important. What are your thoughts on the hype surrounding AI?

JH: It's like AI is both overhyped and underhyped. The issue is that people don't generally understand the opportunities and the constraints, so they think there are opportunities where they don't exist, and they think there are constraints where they don't exist. The latter can really limit your strategy, because you literally aren't aware that there's a whole blind spot in your thinking about how do I create, distribute and sell this product – not realising there's a whole different way of doing it. If there are constraints you're not aware of that can obviously be a huge problem, because you can start going down some path not realising it could never actually work. So I think they're both problems and it really is about getting educated.

AiLab: AI is disrupting all industries and even whole sectors including medicine, transport (autonomous vehicles), agriculture, etc. How do people and organisations identify new opportunities across different sectors?

JH: Yes, it's not a specific sector thing, it's every sector! A good company to study here is Google, because Google got on the deep learning train pretty much the earliest of any of the reasonably big companies. They did something pretty interesting by creating a group called Google Brain, which was their expertise group. They didn’t hide them away in a building out in the corner of the campus, they actually sent the people from Google Brain out to the rest of the organisation and had them sit in with ads, mobile, YouTube, and so on. Which by the way, I think is the right way to do data science in general, not just AI.

Google ended up being a company where not necessarily everybody understood AI, but every group had some experience of building AI into a product. Alongside applied deep learning, this approach enabled them to keep finding ways of doing things that people didn't even realise was a ‘thing’ before. For example, take the mobile experience; Google changed how cameras are made by using cheaper, lower performance sensors, which were made to open for a bit longer. They then used deep learning based techniques (computational photography) to figure out what the pixels probably look like in the low-resolution images. Computational photography can be used in place of huge zoom lenses, to guess what interpolated pixels are required.

I don't know if it's in Australia yet, but here in the USA you call up a phone number for your bank or whatever and when they put you on hold, you can press a button that says, ‘Wait until I'm not on hold anymore and call me back’, and the system uses deep learning to figure out when a real human answers the phone. There's also another deep learning application that will call up a restaurant for you and make a booking. These are all different ways of doing normal stuff, like taking a photo, sitting on hold or making a restaurant booking that utilise AI to do things that maybe a lot of people wouldn't realise were opportunities in the first place.

AiLab: So the opportunities for applied AI are huge?

JH: There's opportunities everywhere! Rather than sector by sector, think of it as being internal to the company. Look at the different verticals and horizontals and how everything is structured in your company. Then think how can HR change? How can logistics change? How can treasury change? Each of the different functions are going to have interesting things that can be done. For example, Google looked at the temperature control of their data centres and wondered if deep learning and reinforcement learning could be used to optimise how that's done automatically, and discovered it would save $3 billion a year — well, that's handy!

AiLab: Do you see deep learning evolving and do you have any thoughts on the future you’d like to share?

JH: It's not really something I spend time thinking about, because in my experience the answer is that nobody knows. There's always a few people who are going to do fundamental research about totally different ways to do things and that's valuable, but it's not who I am. I'm somebody who likes to take something that is already fantastically useful, underused and underappreciated, and help people use it better.

With deep learning we've barely scratched the surface of what it can do, so it seems very premature for too many people spending too much time wondering about what's next. It's an incredibly general framework and I'm sure there's gonna be tweaks; maybe we stop using backpropagation and maybe there are better ways to optimise. Maybe there are steps we can do analytically and maybe standard digital floating point systems are not the right way to to kind of manipulate these things in hardware. These are all pretty big changes, but you could change all of them and it would still clearly be deep learning.

AiLab: As we've barely scratched the surface of what AI and other smart automation techniques can achieve, what impact do you think these technologies will have on the economy and jobs?

JH: This is a key reason why we started fast.ai, because we genuinely believe that there will be haves and have-nots. The organisations and the countries that don't learn about, invest in and use AI will be uncompetitive, just like what happened with the Internet and computers. That worried us because we didn't want to see the inequality in the world increase, and particularly we didn't want to see it increase in such a way that well-off white guys that went to exclusive universities further increased their wealth and the rest of the world became more obsolete. So there are important issues at an organisational level, policy level for countries, and at an individual level with investing in and developing AI expertise.

I talk to a lot of doctors and particularly academic medical researchers across lots of specialties, and they are always asking me, My kid is becoming a doctor so should they learn to code? Should they learn machine learning now?  I think yeah, that business is transforming! Either they're going to be building, designing and setting the path for the tools that are transforming that industry or they're going to be media technicians pressing the buttons they’re told to press.

So yes, it's super important because in the long term we’re automating intellect and perception, which are two things humans are really good at (I said this in my Ted talk back in 2013, which although was quite a while ago, I wouldn't change anything). Something like 80% of developed nation jobs in the service sector are largely pretty basic perception and intellectual tasks. So yes, it's going to have a huge impact on jobs, the economy and society.

AiLab: How do you think this aligns with countries having a national AI strategy and a really solid plan for how to build AI capability, exporting their skillset and products?

JH: So let's talk about Australia in particular. Back in the early days of computing Australia had a chance to have a national computing strategy to be one of the world leaders and they made a conscious decision not to invest. This was followed up with some pretty terrible decisions about the Internet and communications. Luckily, Australia has some brilliant people at places like CSIRO that pretty much invented Wi-Fi, but that was not thanks to a national strategy; it was kind of a lucky break and some brilliant Australian ingenuity. On the whole we've been left behind, as we don't have the biggest and most important companies in the world. For example the USA has Facebook, Apple, Netflix, Google, Amazon, and in China, Alibaba and Tencent.

Australia's just not there, which is such a shame because there's a lot of brilliant research institutions in Australia and there are a lot of brilliant people. Take a look at what China's done; they've made a very conscious decision to put a national AI strategy in place. I talked to the peak body of the Chinese Society of Engineers quite a few years before a national AI strategy was in place and I mentioned they were about to miss out on a huge quantum leap. It's amazing to think that in just eight years they've gone from nothing to pretty much equal leaders in the field. It's not too late for Australia, but it's gonna require a serious change and commitment.

AiLab: There’s been a lot of work around the globe regarding ethics in AI and fast.ai includes discussions on ethics. This topic is so important for machine learning engineers and everyone in AI, can you tell us your thoughts?

JH: As a matter of fact, my wife no longer spends most of her time on fast.ai, because she founded the Center for Applied Data Ethics (CADE) at the University of San Francisco, so it's a big enough deal that one of us is now dedicating their life to it. Anytime you have a technology that is potentially incredibly powerful, then wielding that technology has an impact on humans and on society. If you don't spend time thinking about those impacts, then it's pretty likely that you will end up hurting people.

Most of us actually want to help more people than we hurt and that doesn't just happen magically. A lot of smart people have spent literally hundreds of years thinking about questions around, how do we do the right thing, how do organisations do the right thing, and how do processes create the right things? So there is I think an increasingly small minority of Engineers who seem to genuinely believe the right answer is to put your head in the sand and just say, ‘I'm just an engineer, I don't know anything about this stuff and I shouldn't know anything about this stuff.’ It's kind of madness, right? If you're doing that you're saying I do not have the knowledge about this system to help people use it correctly. If you are the person that literally wrote the system and you don't have that knowledge then who does? I guess in that case the answer is nobody, in which case it's like well, who's job is it to spend the time figuring it out?

To a large degree, the person that wrote the system should be spending some time thinking about that and helping others understand it. If you're the one with the expertise in this system that you helped build or you have expertise in the data that you've helped build it with, then should you spend time learning about the basic foundations of ethical systems, ethical processes and ethical frameworks? If your answer to this is 'no', then I'd say: Do you think it's useful to know how to code?  Yes! Should you spend any time learning how to code properly?  Yes! Do you spend anytime doing math?  Yes! Is it worth spending any time doing and learning how to do math properly?  Yes! Do you spend any time building things that can impact societies?  Yes! Is it worth spending any time figuring out how to build things and impact societies?  No? What? How does that work?

AiLab: For decades some researchers and certain industries have been applying ethical approaches to the work that they do; for example, statisticians in risk assessment. Given this topic isn't exactly new, would you agree that ethical issues aren’t addressed enough in most technology curriculums?

JH: Yes, and I think more generally STEM (Science, Technology, Engineering, and Mathematics) courses could certainly use a deeper integration of sociology, history and psychology. My wife, Rachel Thomas, and I often get invited to panels or think-tank chats about ethics and other related issues. There's often a lot of techie people there who literally invent stuff on the spot and then want to pine on their solutions to ethical quandaries. They don’t realise that literally thousands of people have written tens of thousands of academic papers and have deeply studied these questions already by interviewing people, gathering data, developing hypotheses and testing them.

For example, there is an entire interdisciplinary field of study called Science and Technology Studies (STS), where university groups around the world have for decades studied the question of how science and technology integrate with society. Maybe I'm a bit biased. I have a BA, and my university background for what it's worth was entirely on the arts side. It does drive me a little crazy that a lot of people that work in technology or research in technology, believe correctly, that you can't usefully do stuff in technology if you don't understand technology – yet somehow believe you can usefully do stuff in society if you don't understand sociology.

AiLab: Recently there’s been efforts to produce ethical frameworks around AI. What are your thoughts on the actual application of ethics within an organisation, including monitoring and feedback loops?

JH: It is something we think is important enough that it's chapter 3 of our ‘Deep Learning for Coders with fastai and PyTorch’ book. We try not to think of data ethics as something different, but suggest applying existing knowledge about how you deploy new systems, create new products and enter new markets. There are simple things that can be done like, are there reporting systems in place that indicate if something is going wrong? What kind of things could go wrong? How would that show up if it went wrong? Will you see if that thing shows up? Where exactly are the humans in the loop here? If something does go wrong, is there somebody who is empowered to fix it in an appropriate time frame? So none of these are even technology specific issues. A lot of organisations tend to turn off their brain when it comes to applying ethical AI strategies, because they literally think that they're not smart enough or that it's out of their area of expertise, so they don't do it.

AiLab: I'm excited that you are bringing all your expertise back to Australia. Is there anything you can tell us about your move back here?

JH: Thanks. I'm really excited about it too. I feel like Australia's this huge, greenfield opportunity. Australia has very smart people, great universities, and a science friendly culture; certainly compared to America for example. [Laughs]. It's got a lot of stuff in place and yet, it's nowhere on the world stage when it comes to AI. There are some tiny, but significant pockets of excellence, but sadly except for universities, there's not any organisations that are world leaders. I think there can be and so I'm excited about any opportunities to help make that transition from a high potential place to a high achieving place.

Part of that is self confidence. I don't like how often I see really smart people in Australia doing consulting rather than building a transformative product, for example. It requires a certain level of arrogance to believe you can create a product better than everybody else and that's something I found hard when I last lived in Australia. Although I did build some new products in Australia, the vast majority of people who talked to me about them tried to convince me that other, mainly American companies, were much bigger and much more likely to succeed than I would, so it was pointless me trying. It might even be correct, but it's still not worth saying. [Laughs]

In San Francisco, I have never, ever heard those kinds of comments, whereas in Australia that was what was normal. It’s not to say I'm particularly fond of the American that says, ‘that's totally amazing’, to absolutely everything. I do like the bit of Australian honesty and a bit of a laid back approach, but at the same time I think people require some encouragement. It's scary to build something when your friends and family are telling you can't do it and he shouldn't bother trying, because it's pretty hard to do it anyway.

One thing I want to do is start making some more connections to some of the American academic and industry organisations that have been highly successful, as well as encouraging more intermingling. That might help Australians realise these people who we all hear about as being amazing role models are actually not that amazing, they're just normal people. We might begin to realise Australia is full of brilliant people and we should start creating research centres here, we should start creating recruiting pipelines, we should start creating specialised campuses. I'd love to see that happen and I think it can.

AiLab: That’s awesome! I think having your expertise here will make a huge difference. As you know, the whole AI community in Australia is growing and there is a thirst for change to put Australia on the global AI map. I think your input will only help that, so it's fantastic you’re coming back.

JH: Thanks. I hope so.

AiLab: Thanks so much Jeremy. I really appreciate your time, especially in this time of moving. We're really looking forward to seeing you here in person very soon. Have a great trip!

JH: Thanks John. Fingers crossed will successfully get on a plane tomorrow and successfully get off it again at the other end.

AiLab would like to thank Jeremy for his time and sharing his knowledge and experience with us.

This interview is copyright to koolth pty ltd & AiLab © 2021 and may not be reproduced in whole or part without the express permission of AiLab.