TL;DR Analogies usually help us understand abstract concepts rather than explain their underlying logic. I have therefore always considered the comparison between deep learning and biological neurons rather imprecise, influenced more by the naming than anything else. Reading this book made me realize that their underlying logic is much closer than I had imagined, and that neuroscience has influenced the history of deep learning more deeply than I thought.
Intelligence can be explained, and explained in layers—even along an evolutionary timeline.
Intro This book is structured rather like a review of the literature. When discussing a question, Nick Lane first walks through the existing theories, so part of the pleasure of reading it is following his thinking as he explains a theory and then rules it out. The downside is that it is not always an easy read, and I sometimes lose track of the main argument. Here, then, is a chronological outline of the theory he puts forward.
Intro This post is a set of notes on my understanding of Tailscale’s How NAT traversal works, together with an overview of how several common NAT traversal techniques work.
NAT NAT (Network Address Translation) was introduced primarily to address the shortage of IPv4 addresses.
In simple terms, most machines use a private IP address, such as 192.168.1.2. When a device A with such an address needs to reach a device B on the public Internet, such as 8.
Modernity Much of the confusion surrounding modernity comes from its temporal name: literally, it means being “new.” More specifically, this newness means inaugurating something unprecedented in the flow of time, a novel way of existing. One might even say that its ultimate ambition is not merely a form of being, but a form of becoming.
The dividing line between the ancient and the modern lies in their views of one thing: universals. In the ancient world, realism was the prevailing orthodoxy, holding universals to be ultimate reality.
Intro Let me quote a popular Douban review from more than a decade ago. Reducing the book to this one line does mythologize it a little too much, but it still conveys something of its impact:
There are only two kinds of people in the world: those who have read this book, and those who have not.
To my mind, there are two main reasons readers find it so special:
Its form is extraordinarily elegant and entertaining.
TL;DR To let an application inside a container running on a VM use an NVIDIA GPU:
Pass the host’s GPU through to the VM. Use NVIDIA’s container runtime to expose the GPU inside the container. Use a device plugin for scheduling in Kubernetes. Passthrough in Proxmox VE Configure the Host Add IOMMU Parameters to GRUB Edit the file:
$ vi /etc/default/grub You should find this line:
GRUB_CMDLINE_LINUX_DEFAULT="quiet" Add the Intel or AMD IOMMU parameter, depending on your CPU:
The Four Stages of Compilation What we usually call compilation actually comprises four stages: preprocessing, compilation, assembly, and linking. For example, running gcc without any additional options performs all four in one go, taking a.c through to a.out.
Preprocessing Preprocessing is straightforward: handle all the preprocessing directives, including macro definitions, conditional compilation, and #include. For example, suppose we have this simple program:
#include <stdio.h> int main() { printf("Hello, World!\n"); return 0; } During preprocessing, the #include directive is expanded, inserting the entire stdio header into the current file.
Introducing Assignment None of the Scheme constructs discussed so far modifies an existing value. For example, using define twice for the same identifier produces an error:
(define x 1) (define x 2) ; module: identifier already defined In some situations, however, we really do need to change an existing value. Scheme provides set! for this, allowing us to implement a bank account:
(define balance 100) (define (withdraw amount) (if (>= balance amount) (begin (set! balance (- balance amount)) balance) "Insufficient funds")) A more reasonable design would make balance local to withdraw.
Data Abstraction and Church Numerals Earlier, we built many abstractions around procedures. Data can likewise be organized into layers of abstraction to make complex functionality easier to design.
For example, we can define an abstract data type for rational numbers, represented as fractions:
(define (make-rat n d) (cons n d)) (define (numer x) (car x)) (define (denom x) (cdr x)) We can also define common operations such as addition, subtraction, multiplication, division, and equality:
(define (add-rat x y) (make-rat (+ (* (numer x) (denom y)) (* (numer y) (denom x))) (* (denom x) (denom y)))) (define (sub-rat x y) (make-rat (- (* (numer x) (denom y)) (* (numer y) (denom x))) (* (denom x) (denom y)))) (define (mul-rat x y) (make-rat (* (numer x) (numer y)) (* (denom x) (denom y)))) (define (div-rat x y) (make-rat (* (numer x) (denom y)) (* (denom x) (numer y)))) (define (equal-rat?
The Elements of Programming To construct complex programs, a programming language provides elements at different levels:
Primitive expressions: the smallest entities the language deals with. Means of combination: ways to combine those primitive elements into compound ones. Means of abstraction: ways to name those combinations so they can be used as units. This is abstraction, the central theme of the book. Evaluation Order Combinations follow definite evaluation rules. In Scheme, for example:
(+ 1 2) Here + is the operator, while 1 and 2 are its operands.