Things that I like

Today we’re going to talk about how computers understand speech and speak themselves. As computers play an increasing role in our daily lives there has been an growing demand for voice user interfaces, but speech is also terribly complicated. Vocabularies are diverse, sentence structures can often dictate the meaning of certain words, and computers also have to deal with accents, mispronunciations, and many common linguistic faux pas. The field of Natural Language Processing, or NLP, attempts to solve these problems, with a number of techniques we’ll discuss today. And even though our virtual assistants like Siri, Alexa, Google Home, Bixby, and Cortana have come a long way from the first speech processing and synthesis models, there is still much room for improvement. Produced in collaboration with PBS Digital Studios: http://youtube.com/pbsdigitalstudios Want to know more about Carrie Anne?

The Latest from PBS Digital Studios:

Want to find Crash Course elsewhere on the internet? Facebook -

Twitter - http://www.twitter.com/TheCrashCourse Tumblr - http://thecrashcourse.tumblr.com Support Crash Course on Patreon: http://patreon.com/crashcourse CC Kids: http://www.youtube.com/crashcoursekids

Technovation 2016 Winner Jennifer John introduces Dan Jurafsky, Professor of Linguistics and Computer Science at Stanford University. Dan explains how natural language processing is transforming the way we interact with the world and understand ourselves.

Compiler Explorer is an interactive online compiler which shows the assembly output of compiled C++, Rust, Go (and many more) code.

Copyright ©2015 Emmeline May and Blue Seat Studios Non-commercial use: Video must have copyright information displayed below video, with a live link to original. No alteration to the video may be made, other than translation. Commercial use: Contact [email protected] for licensing. Script - Rockstar Dinosaur Pirate Princess ... Animation - Rachel Brian ... VO - Graham Wheeler http://rockstardinosaurpirateprincess.com/2015/03/02/consent-not-actually-that-complicated/ http://www.blueseatstudios.com/

This blog post should help you understand the major differences considering GCC Vs Clang. Both are excellent software but there are differences to discuss.

Let’s take a brief trip back to our school years and recall some lessons in mathematics and physics. Do you remember what the number π equals? And what is π squared? That’s a strange question too. Of course, it’s 9.87. And do you remember the value of the acceleration due to gravity, g? Of course, that number was drilled into our memory so thoroughly that it’s impossible to forget: 9.81 m/s². Naturally, it can vary, but for solving basic school problems, we typically used this value.

Neural networks have become increasingly impressive in recent years, but there's a big catch: we don't really know what they are doing. We give them data and ways to get feedback, and somehow, they learn all kinds of tasks. It would be really useful, especially for safety purposes, to understand what they have learned and how they work after they've been trained. The ultimate goal is not only to understand in broad strokes what they're doing but to precisely reverse engineer the algorithms encoded in their parameters. This is the ambitious goal of mechanistic interpretability. As an introduction to this field, we show how researchers have been able to partly reverse-engineer how InceptionV1, a convolutional neural network, recognizes images. ▀▀▀▀▀▀▀▀▀SOURCES & READINGS ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ This topic is truly a rabbit hole. If you want to learn more about this important research and even contribute to it, check out this list of sources about mechanistic interpretability and interpretability in general we've compiled for you: On Interpreting InceptionV1: Feature visualization:

Zoom in: An Introduction to Circuits:

The Distill journal contains several articles that try to make sense of how exactly InceptionV1 does what it does:

OpenAI's Microscope tool lets us visualize the neurons and channels of a number of vision models in great detail:

Here's OpenAI's Microscope tool pointed on layer Mixed3b in InceptionV1:

Activation atlases:

More recent work applying SAEs to InceptionV1:

Transformer Circuits Thread, the spiritual successor of the circuits thread on InceptionV1. This time on transformers:

In the video, we cite "Toy Models of Superposition":

We also cite "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning":

More recent progress: Mapping the Mind of a Large Language Model: Press:

Paper in the transformers circuits thread:

Extracting Concepts from GPT-4: Press:

Paper:

Browse features:

Language models can explain neurons in language models (cited in the video): Press:

Paper:

View neurons:

Neel Nanda on how to get started with Mechanistic Interpretability: Concrete Steps to Get Started in Transformer Mechanistic Interpretability:

Mechanistic Interpretability Quickstart Guide:

200 Concrete Open Problems in Mechanistic Interpretability:

More work mentioned in the video: Progress measures for grokking via mechanistic interpretability:

Discovering Latent Knowledge in Language Models Without Supervision:

Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning:

▀▀▀▀▀▀▀▀▀PATREON, MEMBERSHIP, MERCH ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🟠 Patreon:

Transformer based models like LLMs have demonstrated remarkable prowess in natural language processing tasks. However, their limitations…