Exploring the Power of Neural Networks: How They Learn and Why They are Useful

How neural networks can (almost!) learn anything
Video about neural networks and how they work.

My twitter: https://twitter.com/max_romana.

SOURCES
Neural network playground: https://playground.tensorflow.org/

Universal Function Approximation
Proof: https://cognitivemedium.com/magic_paper/assets/Hornik.pdf.
Covering ReLUs: https://proceedings.neurips.cc/paper/2017/hash/32cbf687880eb…tract.html.
Covering discontinuous functions: https://arxiv.org/pdf/2012.03016.pdf.

Turing Completeness:
The networks of infinite size have been completed: Neural Computing I & II. (Unfortunately behind a paywall, but cited in the following paper.)
RNNs are turing complete: https://binds.cs.umass.edu/papers/1992_Siegelmann_COLT.pdf.
Transformers are turing complete: https://arxiv.org/abs/2103.

Backpropagation in more detail:

Source: