NumPy vs APL¶

https://youtu.be/ZB0IJMRpbRE

@MaxCan-Code "Max Sun APL"

NumPy vs APL¶

  • Going slow and easy, we have an hour
  • APL NumPy side-by-side
  • Goal: aware of APL as another tool
  • Non-goal: switch
  • Cover briefly then come back for Qs
  • Intentionally left out details so please ask as I go

About me¶

  • Applied math in school
  • Knew APL from YouTube
  • Learned APL on the job last Jan (https://bcaresearch.com customer of Dyalog)

What we'll do¶

  1. Simple Moore's Law model
  2. Real data (CSV)
  3. Perform linear regression and predict exponential growth using ordinary least squares (tldr line of best fit)
  • Compare exponential growth constants between models
  • Save and share:
    • NumPy zipped file *.npz
    • APL namespace file *.apln
    • CSV
  • Chat, happy little demos?

Modified from https://numpy.org/numpy-tutorials/content/mooreslaw-tutorial.html

What we need¶

  • Python (packages):
    • NumPy
    • Matplotlib
  • Dyalog APL:
    • SharpPlot (included) (OOP)
    • Py'n'APL (optional, external, open source)

APL trigger warning

  • Butchered NumPy-style APL for comparison
  • APL users, sorry
In [1]:
import matplotlib.pyplot as plt
import numpy as np
from pynapl import APL
apl = APL.APL()
run = apl.eval

Dyalog APL/S-64 Version 19.0.48958
Serial number: UNREGISTERED - not for commercial use
+-----------------------------------------------------------------+
| Dyalog is free for non-commercial use but is not free software. |
| A basic licence can be used for experiments and proof of        |
| concept until the point in time that it is of value.            |
| For further information visit                                   |
| https://www.dyalog.com/prices-and-licences.htm                  |
+-----------------------------------------------------------------+
Copyright (c) Dyalog Limited 1982-2024

      
          ⎕PW←32767
          {}2⎕FIX'file:///home/user/.cache/pypoetry/virtualenvs/non-package-mode-gv7LMvVc-py3.11/lib/python3.11/site-packages/pynapl/IPC.dyalog'
          {}2⎕FIX'file:///home/user/.cache/pypoetry/virtualenvs/non-package-mode-gv7LMvVc-py3.11/lib/python3.11/site-packages/pynapl/Py.dyalog'
          infile←'/tmp/tmpvmuf50n2'
          outfile←'/tmp/tmpxqqgclxh'
          Py.StartAPLSlave infile outfile

Model¶

Moores_law(year) = (e^B_M) × (e^(A_M × year))

A_M = (log 2) ÷ 2

B_M = (log 2250) - A_M × 1971

In [2]:
     #        exp←*
     #        log←*○
     A_M = np.log(2) / 2
run("A_M ←     (⍟ 2) ÷ 2")

     B_M = np.log(2250) - A_M * 1971
run("B_M ←     (⍟ 2250) - A_M × 1971")

     Moores_law = lambda year: np.exp(B_M) * np.exp(A_M * year)
run("Moores_law ← {      year←⍵ ⋄  (* B_M) ×      *(A_M × year)} ⋄")

[A_M, B_M], run("A_M B_M")
Out[2]:
([0.34657359027997264, -675.3778609466276],
 [0.3465735902799726, -675.3778609466276])

In 1971, there were 2250 transistors. Use Moores_law to predict how many in 1973.

In [3]:
ML_1971 = Moores_law(1971)
ML_1973 = Moores_law(1973)
print("In 1973, G. Moore expects {:.0f} transistors on Intels chips".format(ML_1973))
print("This is x{:.2f} more transistors than 1971".format(ML_1973 / ML_1971))

In 1973, G. Moore expects 4500 transistors on Intels chips
This is x2.00 more transistors than 1971
In [4]:
run("""
ML_1971←Moores_law 1971
ML_1973←Moores_law 1973

_←'.format←⍕'
⎕←'In 1973, G. Moore expects ',(1↓0⍕ML_1973),' transistors on Intels chips'
⎕←'This is x',(1↓2⍕ML_1973÷ML_1971),' more transistors than 1971'
""");

In 1973, G. Moore expects 4500 transistors on Intels chips
This is x2.00 more transistors than 1971

Loading historical data¶

  • Transistor count each year in CSV
  • Inspect before load
  • Save year & count columns to data.

Print first 10 rows of transistor_data.csv. The columns are

Processor MOS transistor count Date of Introduction Designer MOSprocess Area
Intel 4004 (4-bit 16-pin) 2250 1971 Intel "10,000 nm" 12 mm²
... ... ... ... ... ...
In [5]:
! head transistor_data.csv

Processor,MOS transistor count,Date of Introduction,Designer,MOSprocess,Area
Intel 4004 (4-bit  16-pin),2250,1971,Intel,"10,000 nm",12 mm²
Intel 8008 (8-bit  18-pin),3500,1972,Intel,"10,000 nm",14 mm²
NEC μCOM-4 (4-bit  42-pin),2500,1973,NEC,"7,500 nm",?
Intel 4040 (4-bit  16-pin),3000,1974,Intel,"10,000 nm",12 mm²
Motorola 6800 (8-bit  40-pin),4100,1974,Motorola,"6,000 nm",16 mm²
Intel 8080 (8-bit  40-pin),6000,1974,Intel,"6,000 nm",20 mm²
TMS 1000 (4-bit  28-pin),8000,1974,Texas Instruments,"8,000 nm",11 mm²
MOS Technology 6502 (8-bit  40-pin),4528,1975,MOS Technology,"8,000 nm",21 mm²
Intersil IM6100 (12-bit  40-pin; clone of PDP-8),4000,1975,Intersil,,
In [6]:
run("""
⎕←↑'UTF-8'∘⎕UCS¨⎕UCS¨⎕SH'head transistor_data.csv'
⎕←''
⎕←↑⎕SH'head transistor_data.csv'
""");

Processor,MOS transistor count,Date of Introduction,Designer,MOSprocess,Area  
Intel 4004 (4-bit  16-pin),2250,1971,Intel,"10,000 nm",12 mm²                 
Intel 8008 (8-bit  18-pin),3500,1972,Intel,"10,000 nm",14 mm²                 
NEC μCOM-4 (4-bit  42-pin),2500,1973,NEC,"7,500 nm",?                         
Intel 4040 (4-bit  16-pin),3000,1974,Intel,"10,000 nm",12 mm²                 
Motorola 6800 (8-bit  40-pin),4100,1974,Motorola,"6,000 nm",16 mm²            
Intel 8080 (8-bit  40-pin),6000,1974,Intel,"6,000 nm",20 mm²                  
TMS 1000 (4-bit  28-pin),8000,1974,Texas Instruments,"8,000 nm",11 mm²        
MOS Technology 6502 (8-bit  40-pin),4528,1975,MOS Technology,"8,000 nm",21 mm²
Intersil IM6100 (12-bit  40-pin; clone of PDP-8),4000,1975,Intersil,,         

Processor,MOS transistor count,Date of Introduction,Designer,MOSprocess,Area     
Intel 4004 (4-bit  16-pin),2250,1971,Intel,"10,000 nm",12 mm²                 
Intel 8008 (8-bit  18-pin),3500,1972,Intel,"10,000 nm",14 mm²                 
NEC μCOM-4 (4-bit  42-pin),2500,1973,NEC,"7,500 nm",?                          
Intel 4040 (4-bit  16-pin),3000,1974,Intel,"10,000 nm",12 mm²                 
Motorola 6800 (8-bit  40-pin),4100,1974,Motorola,"6,000 nm",16 mm²            
Intel 8080 (8-bit  40-pin),6000,1974,Intel,"6,000 nm",20 mm²                  
TMS 1000 (4-bit  28-pin),8000,1974,Texas Instruments,"8,000 nm",11 mm²        
MOS Technology 6502 (8-bit  40-pin),4528,1975,MOS Technology,"8,000 nm",21 mm²
Intersil IM6100 (12-bit  40-pin; clone of PDP-8),4000,1975,Intersil,,

Only need columns 2 & 3

np.loadtxt:

  • delimiter = ','
  • usecols = [1,2]: import cols 2 & 3
  • skiprows = 1: skip header row
In [7]:
data = np.loadtxt("transistor_data.csv", delimiter=",", usecols=[1, 2], skiprows=1)
In [8]:
run("""
csv←(⎕CSV⍠'Separator' ',')'transistor_data.csv'⍬ 4
data←1↓csv[;2 3]
data←1⊃⎕CSV'transistor_data.csv'⍬(0 4 4 0 0 0)1
""");

# csv = np.loadtxt("transistor_data.csv") # Value Error: could not convert string to float
# data = csv[1:, 1:3]
  • Assign data to year and transistor_count
  • Print first 10 values ([:10])
In [9]:
year = data[:, 1]  # grab the second column and assign
transistor_count = data[:, 0]  # grab the first column and assign

print("year:\t\t", year[:10])
print("trans. cnt:\t", transistor_count[:10])

year:		 [1971. 1972. 1973. 1974. 1974. 1974. 1974. 1975. 1975. 1975.]
trans. cnt:	 [2250. 3500. 2500. 3000. 4100. 6000. 8000. 4528. 4000. 5000.]
In [10]:
run("""
year←data[;2]⊣'  ⍝ grab the second column and assign'
transistor_count←data[;1]⊣'  ⍝ grab the first column and assign'

⎕←'year:		',10↑year
⎕←'trans. cnt:	',10↑transistor_count ⋄
""");

year:		 1971 1972 1973 1974 1974 1974 1974 1975 1975 1975
trans. cnt:	 2250 3500 2500 3000 4100 6000 8000 4528 4000 5000

Goal: solve yi = A × year + B for A, B

In [11]:
yi = np.log(transistor_count)
run("yi←  ⍟ transistor_count");

Use least squares

In [12]:
model = np.polynomial.Polynomial.fit(year, yi, deg=1)
model = model.convert()
model
Out[12]:
$x \mapsto \text{-666.32640635} + \text{0.34163208}\,x$
In [13]:
run("""
_←'https://aplcart.info?q=linear%20fit'
_←'matrix div ⌹ ⎕÷'
model←yi⌹(1,⍪year)
(B A)←model
⎕←'x' ' ' '↦'(1↓9⍕B)'+'(1↓9⍕A)'x'
""");

x ↦ ¯666.326406352 + 0.341632083 x

The individual parameters A and B are the coefficients of our linear model:

In [14]:
B, A = model
# (B A)←model

Increase rate = e^(2 × A)

In [15]:
print(f"Rate of semiconductors added on a chip every 2 years: {np.exp(2 * A):.2f}")

Rate of semiconductors added on a chip every 2 years: 1.98
In [16]:
run("""
⎕←'Rate of semiconductors added on a chip every 2 years: ',1↓2⍕*2×A
""");

Rate of semiconductors added on a chip every 2 years: 1.98

Plot & compare:

  • Least squares
  • Moore's law
  • Real world data

style sheet: fivethirtyeight

In [17]:
transistor_count_predicted = np.exp(B) * np.exp(A * year)
transistor_Moores_law = Moores_law(year)
plt.style.use("fivethirtyeight")
plt.semilogy(year, transistor_count, "s", label="MOS transistor count")
plt.semilogy(year, transistor_count_predicted, label="linear regression")


plot1 = plt.plot(year, transistor_Moores_law, label="Moore's Law")
plt.title(
    "MOS transistor count per microprocessor\n"
    + "every two years \n"
    + "Transistor count was x{:.2f} higher".format(np.exp(A * 2))
)
plt.xlabel("year introduced")
plt.legend(loc="center left", bbox_to_anchor=(1, 0.5))
plt.ylabel("# of transistors\nper microprocessor")
Out[17]:
Text(0, 0.5, '# of transistors\nper microprocessor')
No description has been provided for this image
In [18]:
plot1[0].figure
Out[18]:
No description has been provided for this image
In [19]:
run("""
_←']Chart ⍝ useful, windows only'
_538←{_sp←⍵
    _←_sp.SetLineStyles do Causeway.LineStyle.Solid
    FromHtml←Causeway.ColorTranslator.{FromHtml'#',⍵}
    _←'FromHtml←{256⊥255,16⊥⍉3 2⍴¯1+⍵⍳⍨⎕D,⎕C ⎕A}'
    _←'https://github.com/matplotlib/matplotlib/blob/main/lib/matplotlib/mpl-data/stylelib/fivethirtyeight.mplstyle'
    _←_sp.SetColors do⊂Causeway.{Color.(⍵)}FromHtml¨'008fd5' 'fc4f30' 'e5ae38' '6d904f' '8b8b8b' '810f7c'
    _←_sp.SetBackground do Causeway.{Color.(⍵)}FromHtml'f0f0f0'
    _←_sp.SetKeyBackground do Causeway.{Color.(⍵)}FromHtml'f0f0f0'
    _←_sp.SetAxisStyle do(Causeway.{Color.(⍵)}FromHtml'cbcbcb')Causeway.LineStyle.Solid 0.5
    _←_sp.SetGridLineStyle do(Causeway.{Color.(⍵)}FromHtml'cbcbcb')Causeway.LineStyle.Solid 0.5
    _←_sp.SetPenWidths do 3
    font←'DejaVu Sans'
    _←_sp.SetHeadingFont do font
    _←_sp.SetCaptionFont do font
    _←_sp.SetLabelFont do font
    _←_sp.SetKeyFont do font
    _sp.KeyStyle←Causeway.KeyStyles.(Boxed+Vertical+MiddleAlign)
    _sp.XAxisStyle←Causeway.XAxisStyles.CenteredCaption
    _sp.YAxisStyle←Causeway.YAxisStyles.CenteredCaption
    _sp.YLabelFormat←'0.0E00'
    _sp.ScatterPlotStyle←Causeway.ScatterPlotStyles.GridLines
    _sp.LineGraphStyle←Causeway.LineGraphStyles.GridLines
    _sp}
do←{⍎'⍺⍺ ⍵ ⋄ 0' ⋄ ⍺⍺}
'InitCauseway'⎕CY'sharpplot'
InitCauseway ⍬
sp←⎕NEW Causeway.SharpPlot
""");
In [20]:
import IPython
IPython.display.SVG(run("""
transistor_count_predicted←(*B)×(*A×year)
transistor_Moores_law←Moores_law year
sp←_538 sp
_←sp.SetKeyText do'MOS transistor count' 'linear regression' 'Moore''s Law'
sp.Heading←'MOS transistor count per microprocessor',(⎕UCS 10),'every two years',(⎕UCS 10),'Transistor count was x ',(1↓2⍕*A×2),' higher'
sp.XCaption←'year introduced'
sp.YCaption←'# of transistors',(⎕UCS 10),'per microprocessor'
sp.YAxisStyle←sp.YAxisStyle+Causeway.YAxisStyles.LogScale
sp.KeyStyle←sp.KeyStyle+Causeway.KeyStyles.RightAlign
_←sp.SetMarkers do Causeway.Marker.Block
_←sp.SetMargins do 80 30 50 150
_←sp.SetXTickMarks do 10
_←sp.SetYTickMarks do 2
_←sp.DrawScatterPlot do(transistor_count)year
_←sp.DrawLineGraph do transistor_count_predicted year
_←sp.DrawLineGraph do transistor_Moores_law year
sp.RenderSvg ⍬
"""))
Out[20]:
No description has been provided for this image
In [21]:
IPython.display.SVG(run("sp.RenderSvg ⍬"))
Out[21]:
No description has been provided for this image

Zoom in on year == 2017, compare:

  • Moore's law
  • Average transistor count
  • Real world data

Use alpha=0.2, opaque points mean overlap

In [22]:
transistor_count2017 = transistor_count[year == 2017]
print(
    transistor_count2017.max(), transistor_count2017.min(), transistor_count2017.mean()
)
y = np.linspace(2016.5, 2017.5)
your_model2017 = np.exp(B) * np.exp(A * y)
Moore_Model2017 = Moores_law(y)

19200000000.0 250000000.0 7050000000.0
In [23]:
run("""
transistor_count2017←transistor_count[⍸year=2017]
_←'transistor_count2017←(year=2017)/transistor_count'
mean←{v←⍵ ⋄ (+/v)÷(≢v)}
⎕←(⌈/transistor_count2017)(⌊/transistor_count2017)(mean transistor_count2017)

y←2016.5 2017.5
your_model2017←(*B)×(*A×y)
Moore_Model2017←Moores_law y
""");

1.92E10 250000000 7050000000
In [24]:
plt.plot(
    2017 * np.ones(np.sum(year == 2017)),
    transistor_count2017,
    "ro",
    label="2017",
    alpha=0.2,
)
plt.plot(2017, transistor_count2017.mean(), "g+", markersize=20, mew=6)

plt.plot(y, your_model2017, label="Your prediction")
plot2 = plt.plot(y, Moore_Model2017, label="Moores law")
plt.ylabel("# of transistors\nper microprocessor")
plt.legend()
Out[24]:
<matplotlib.legend.Legend at 0x7f35a9700a50>
No description has been provided for this image
In [25]:
plot2[0].figure
Out[25]:
No description has been provided for this image
In [26]:
IPython.display.SVG(run("""
_←sp.Reset do ⍬
sp←_538 sp
_←sp.SetXRange do y
_←sp.SetYRange do⍎'2.4e10'
_←sp.SetXTickMarks do 0.2
_←sp.SetMarkers do⊂Causeway.Marker.(Dot Plus)
_←sp.SetMarkerColors do Causeway.Color.LightCoral
_←sp.SetMarkerScales do 6
_←sp.SetMargins do 5 15 55 20
_←sp.SetYTickMarks do⍎'0.5e10'
sp.YCaption←'# of transistors',(⎕UCS 10),'per microprocessor'
_←sp.DrawScatterPlot do(transistor_count2017)(2017×(+/year=2017)⍴1)
_←sp.SetMarkerColors do Causeway.Color.Green
_←sp.SetPenWidths do 4
_←sp.SetMarkerScales do 3
_←sp.SetKeyText do'2017' ' ' 'Your prediction' 'Moore''s Law'
_←sp.DrawScatterPlot do,¨(mean transistor_count2017)2017
sp←_538 sp
sp.KeyStyle←sp.KeyStyle+Causeway.KeyStyles.BottomAlign
_←sp.DrawLineGraph do your_model2017 y
_←sp.DrawLineGraph do Moore_Model2017 y
sp.RenderSvg ⍬
"""))
Out[26]:
No description has been provided for this image
In [27]:
IPython.display.SVG(run("sp.RenderSvg ⍬"))
Out[27]:
No description has been provided for this image
  • Least squares: close to mean
  • Moore's law: close to max
  • Least squares: close to mean
  • Moore's law: close to max

Share as zipped arrays and CSV¶

  • np.savez: NumPy arrays for other Python sessions
  • APL namespace (also possible: component file)
  • np.savetxt, ⎕CSV: CSV

savez with notes=notes

In [28]:
notes = "the arrays in this file are the result of a linear regression model\n"
notes += "the arrays include\nyear: year of manufacture\n"
notes += "transistor_count: number of transistors reported by manufacturers in a given year\n"
notes += "transistor_count_predicted: linear regression model = exp({:.2f})*exp({:.2f}*year)\n".format(
    B, A
)
notes += "transistor_Moores_law: Moores law =exp({:.2f})*exp({:.2f}*year)\n".format(
    B_M, A_M
)
notes += "regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B"
print(notes)

the arrays in this file are the result of a linear regression model
the arrays include
year: year of manufacture
transistor_count: number of transistors reported by manufacturers in a given year
transistor_count_predicted: linear regression model = exp(-666.33)*exp(0.34*year)
transistor_Moores_law: Moores law =exp(-675.38)*exp(0.35*year)
regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B
In [29]:
run("""
nl←⎕UCS 10
notes←'the arrays in this file are the result of a linear regression model',nl
notes,←'the arrays include',nl,'year: year of manufacture',nl
notes,←'transistor_count: number of transistors reported by manufacturers in a given year',nl
notes,←'transistor_count_predicted: linear regression model = exp(',(1↓2⍕B),')*exp(',(1↓2⍕A),'*year)',nl

notes,←'transistor_Moores_law: Moores law =exp(',(1↓2⍕B_M),')*exp(',(1↓2⍕A_M),'*year)',nl

notes,←'regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B'
⎕←notes
""");

the arrays in this file are the result of a linear regression model
the arrays include
year: year of manufacture
transistor_count: number of transistors reported by manufacturers in a given year
transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year)
transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year)
regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B
In [30]:
np.savez(
    "mooreslaw_regression.npz",
    notes=notes,
    year=year,
    transistor_count=transistor_count,
    transistor_count_predicted=transistor_count_predicted,
    transistor_Moores_law=transistor_Moores_law,
    regression_csts=(A, B),
)
In [31]:
run("""
_ns←⎕NS'notes' 'year' 'transistor_count' 'transistor_count_predicted' 'transistor_Moores_law'
_ns.regression_csts←A B
_←'https://dyalog.github.io/link/4.0/API/Link.Fix'
_←(⊂⎕SE.Dyalog.Array.Serialise _ns)⎕NPUT'mooreslaw_regression.apln' 1

_←'"" "mooreslaw_regression"⎕SE.Link.Fix ⎕SE.Dyalog.Array.Serialise _ns'
""");
In [32]:
results = np.load("mooreslaw_regression.npz")
run("""
results←⎕SE.Dyalog.Array.Deserialise⊃⎕NGET'mooreslaw_regression.apln'
""");
In [33]:
print(      results["regression_csts"][1])
run("⎕←1↓13⍕results. regression_csts  [2]");

-666.3264063536233
¯666.3264063521225
In [34]:
! ls

air-quality-data.csv
mooreslaw_regression.apln
mooreslaw_regression.csv
mooreslaw_regression.npz
mooreslaw-tutorial.ipynb
mooreslaw-tutorial.md
pairing.md
quadna.dws
save-load-arrays.md
_static
text_preprocessing.py
transistor_data.csv
tutorial-air-quality-analysis.md
tutorial-deep-learning-on-mnist.md
tutorial-deep-reinforcement-learning-with-pong-from-pixels.md
tutorial-ma.md
tutorial-nlp-from-scratch
tutorial-nlp-from-scratch.md
tutorial-plotting-fractals
tutorial-plotting-fractals.md
tutorial-static_equilibrium.md
tutorial-style-guide.md
tutorial-svd.md
tutorial-x-ray-image-processing
tutorial-x-ray-image-processing.md
who_covid_19_sit_rep_time_series.csv
In [35]:
run("⎕←↑⎕SH'ls'");

air-quality-data.csv                                         
mooreslaw_regression.apln                                    
mooreslaw_regression.csv                                     
mooreslaw_regression.npz                                     
mooreslaw-tutorial.ipynb                                     
mooreslaw-tutorial.md                                        
pairing.md                                                   
quadna.dws                                                   
save-load-arrays.md                                          
_static                                                      
text_preprocessing.py                                        
transistor_data.csv                                          
tutorial-air-quality-analysis.md                             
tutorial-deep-learning-on-mnist.md                           
tutorial-deep-reinforcement-learning-with-pong-from-pixels.md
tutorial-ma.md                                               
tutorial-nlp-from-scratch                                    
tutorial-nlp-from-scratch.md                                 
tutorial-plotting-fractals                                   
tutorial-plotting-fractals.md                                
tutorial-static_equilibrium.md                               
tutorial-style-guide.md                                      
tutorial-svd.md                                              
tutorial-x-ray-image-processing                              
tutorial-x-ray-image-processing.md                           
who_covid_19_sit_rep_time_series.csv

CSV¶

  • np.savetxt with header=head
  • need 2D array
In [36]:
head = "the columns in this file are the result of a linear regression model\n"
head += "the columns include\nyear: year of manufacture\n"
head += "transistor_count: number of transistors reported by manufacturers in a given year\n"
head += "transistor_count_predicted: linear regression model = exp({:.2f})*exp({:.2f}*year)\n".format(
    B, A
)
head += "transistor_Moores_law: Moores law =exp({:.2f})*exp({:.2f}*year)\n".format(
    B_M, A_M
)
head += "year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:"
print(head)

the columns in this file are the result of a linear regression model
the columns include
year: year of manufacture
transistor_count: number of transistors reported by manufacturers in a given year
transistor_count_predicted: linear regression model = exp(-666.33)*exp(0.34*year)
transistor_Moores_law: Moores law =exp(-675.38)*exp(0.35*year)
year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:
In [37]:
run("""
head←'the columns in this file are the result of a linear regression model',nl
head,←'the columns include',nl,'year: year of manufacture',nl
head,←'transistor_count: number of transistors reported by manufacturers in a given year',nl
head,←'transistor_count_predicted: linear regression model = exp(',(1↓2⍕B),')*exp(',(1↓2⍕A),'*year)',nl

head,←'transistor_Moores_law: Moores law =exp(',(1↓2⍕B_M),')*exp(',(1↓2⍕A_M),'*year)',nl

head,←'year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:'
⎕←head
""");

the columns in this file are the result of a linear regression model
the columns include
year: year of manufacture
transistor_count: number of transistors reported by manufacturers in a given year
transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year)
transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year)
year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:

Build 2D array by np.blocking 1D vectors with np.newaxis (1-col arrays) together

In [38]:
print(  year.shape)
run("⎕←⍴year");

print(   year[:,np.newaxis].shape)
run("⎕←⍴⍪year");

(179,)
179
(179, 1)
179 1
In [39]:
output = np.block(
    [
        year[:, np.newaxis],
        transistor_count[:, np.newaxis],
        transistor_count_predicted[:, np.newaxis],
        transistor_Moores_law[:, np.newaxis],
    ]
)
In [40]:
run("""
output←year,transistor_count,transistor_count_predicted,⍪transistor_Moores_law
""");
In [41]:
np.savetxt("mooreslaw_regression.csv", X=output, delimiter=",", header=head)
In [42]:
! head mooreslaw_regression.csv

# the columns in this file are the result of a linear regression model
# the columns include
# year: year of manufacture
# transistor_count: number of transistors reported by manufacturers in a given year
# transistor_count_predicted: linear regression model = exp(-666.33)*exp(0.34*year)
# transistor_Moores_law: Moores law =exp(-675.38)*exp(0.35*year)
# year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:
1.971000000000000000e+03,2.250000000000000000e+03,1.130514785642591733e+03,2.249999999999916326e+03
1.972000000000000000e+03,3.500000000000000000e+03,1.590908400344571419e+03,3.181980515339620069e+03
1.973000000000000000e+03,2.500000000000000000e+03,2.238793840142739555e+03,4.500000000000097316e+03
In [43]:
run("""
content←⊂(('^'⎕R'# &')head),nl,output(⎕CSV⍠'Separator' ',')''
_←content ⎕NPUT'mooreslaw_regression.csv' 1
""");
In [44]:
run("""
⎕←↑'UTF-8'∘⎕UCS¨⎕UCS¨⎕SH'head mooreslaw_regression.csv'
⎕←''
⎕←↑⎕SH'head mooreslaw_regression.csv'
""");

# the columns in this file are the result of a linear regression model             
# the columns include                                                              
# year: year of manufacture                                                        
# transistor_count: number of transistors reported by manufacturers in a given year
# transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year)
# transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year)                   
# year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:    
1971,2250,1130.514785668682,2249.999999999916                                      
1972,3500,1590.908400380202,3181.98051533962                                       
1973,2500,2238.793840191098,4500.000000000097                                      

# the columns in this file are the result of a linear regression model              
# the columns include                                                               
# year: year of manufacture                                                         
# transistor_count: number of transistors reported by manufacturers in a given year 
# transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year)
# transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year)                   
# year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:     
1971,2250,1130.514785668682,2249.999999999916                                       
1972,3500,1590.908400380202,3181.98051533962                                        
1973,2500,2238.793840191098,4500.000000000097

Functions we used¶
np.loadtxt ←→ ⎕CSV [;2 3] 1↓
np.log ←→ ⍟
np.exp ←→ *
lambda ←→ {⍵}
plt.semilogy ←→ Causeway.YAxisStyles.LogScale
plt.plot ←→ sp.RenderSvg, sp.Draw* (OOP)
x[:10] ←→ 10↑x
x[year == 2017] ←→ (year=2017)/x or x[⍸year=2017]
np.block ←→ , ⍪
np.newaxis ←→ ⍪
np.savez ←→ ⎕SE.Dyalog.Array.Serialise ⎕NPUT

Some points¶

  • Specific functions ←→ small composable functions (FP)
  • Composing symbols: ⍟ ○* ⌹ ⎕÷
  • Nested function calls ←→ shallow definitions (debugging context)
  • Mature ecosystem (plot styles, exports) ←→ workarounds/hacks/rolling your own

Some points¶

  • Specific functions ←→ small composable functions (FP)
  • Composing symbols: ⍟ ○* ⌹ ⎕÷
  • Nested function calls ←→ shallow definitions (debugging context)
  • Mature ecosystem (plot styles, exports) ←→ workarounds/hacks/rolling your own

Chat, Q&A¶

happy little demos?

  • Point free appendix
  • 1-liner glass cleaner fractal
  • Open emacs from RIDE, prefix completion, d.apln jump
  • XML: 1-liner ]Defs workflow, https://apl.quest vids

@MaxCan-Code "Max Sun APL"

Point-free (tacit) demo¶

  • Opinionated takes
  • Not liable for mental dmg
  • n+2 letter name/notation/definition/formula/src code?
  • see as 1 unit (poly morph ism ←→ polymorphism):
    • (B_M×⍨∘*⍨∘*A_M×⊢)
  • vs autocomplete
In [45]:
Moores_law = lambda year: np.exp(B_M) * np.exp(A_M * year)
Moores_law(1971), Moores_law(1973)
Out[45]:
(2249.9999999999163, 4500.000000000097)
In [46]:
run("""
_←'Moores_law = lambda year: np.exp(B_M) * np.exp(A_M * year)'
   Moores_law ← {      year←⍵ ⋄  (* B_M) ×      *(A_M × year)}
⎕←                                 (B_M  ×⍨∘*⍨∘ * A_M × ⊢)1971 1973
_←'                                └────────┘'

⎕←(B_M×⍨∘*⍨∘*A_M×⊢)1971 1973
⎕←Moores_law       1971 1973

_←'https://aplcart.info?q=split%20compose (looks better next ver)'
""");

2250 4500
2250 4500
2250 4500
  • n+2 letter name/notation/definition/formula/src code?
  • see as 1 unit (poly morph ism ←→ polymorphism):
    • (⊣⌹1,∘⍪⊢)
  • vs autocomplete
In [47]:
model = np.polynomial.Polynomial.fit(year, yi, deg=1)
model = model.convert()
B, A = model
print(B ,A)

-666.3264063536233 0.3416320825591619
In [48]:
run("""
⎕←yi⌹(1,⍪year)

⎕←yi  ⌹1, ⍪  year
⎕←yi(⊣⌹1,∘⍪⊢)year
""");

¯666.3264064 0.3416320826
¯666.3264064 0.3416320826
¯666.3264064 0.3416320826
  • n+2 letter name/notation/definition/formula/src code?
  • see as 1 unit (poly morph ism ←→ polymorphism):
    • (⌈/ , ⌊/ , +/÷≢)
  • vs autocomplete
In [49]:
print(
    transistor_count2017.max(), transistor_count2017.min(), transistor_count2017.mean()
)

19200000000.0 250000000.0 7050000000.0
In [50]:
run("""
mean←{v←⍵ ⋄ (+/v)÷(≢v)}
⎕←(⌈/transistor_count2017)(⌊/transistor_count2017)(mean transistor_count2017)
⎕←(⌈/       ,              ⌊/            ,         +/÷≢) transistor_count2017

⎕←(⌈/,⌊/,+/÷≢)transistor_count2017
""");

1.92E10 250000000 7050000000
1.92E10 250000000 7050000000
1.92E10 250000000 7050000000

Related Work¶

  • Alan J. Perlis. 1977. In praise of APL: a language for lyrical programming. SIGAPL APL Quote Quad 8, 2 (December 1977), 44–47. https://doi.org/10.1145/586015.586019
  • J: primary parts of speech:
noun ←→ data
verb ←→ function on nouns
adverb ←→ modify verb

https://code.jsoftware.com/wiki/Vocabulary/Words#Parts_Of_Speech