create-wagmi cli to start the app. You should see something like this:\n![](\"https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F889302f3-be3b-485d-a4ff-59be5aefc274_1108x720.gif\")
wagmi + rainbow <3Web scraping has always been fascinating to me – the thrill of harvesting forbidden fruit.
Today I learned just how much things have advanced.
python requests-issued HTTP request and the fingerprint you'd expect from the request's User-Agent.For more specifics on the techniques each company uses, this AI chat is fairly comprehensive.
I don't see how this ends besides a GAN-type war between model apps (e.g. Firecrawl), who rely on sophisticated, dubiously legal scraping pipelines, and security providers (Akamai & Cloudflare), who sell millions of enterprise contracts on protecting your data from relentless bot armies.
Unlike most other software battles, I see the odds being in favor of offense. Realistically, bot's will be able to mimic human browsing behavior perfectly, to the point here someone who really wants to protect their data from scraping needs to put the bulk of it behind a paywall or a paid API. Do you see Amazon.com doing that, dropping conversion rates, and evaporating shareholder value? No chance. But some more boutique websites definitely could. Even then, there will likely still be profit to be had at an aggregation layer which subscribes to expensive data behind paywalls, builds value on top, and captures value internally or exposes a new, higher-level platform which charges it's own spread.
For these reasons, I'm glad I upped my scraping game. It will be a valuable skill for a long time to come.
", "url": "https://whoislewys.com/blog/2026/06/25/til-the-state-of-the-art-in-the-scraper-vs-bot-detection-arms-race", "title": "TIL: The State of the Art in the Scraper VS. Bot-Detection Arms Race", "summary": "Web scraping has always been fascinating to me – the thrill of harvesting forbidden fruit.", "date_modified": "2026-06-25T00:00:00.000Z", "author": { "name": "Luis Gomez", "url": "https://github.com/whoislewys" }, "tags": [ "scraping", "til" ] }, { "id": "https://whoislewys.com/blog/2026/03/08/ai-assisted-jupyter-notebook-development", "content_html": "I discovered some extremely effective ways to work with Jupyter Notebooks with LLM assistance this week.
Things I found massively helpful:
Every Jupyter Notebook is just a JSON file with a specific structure. Each cell in a Jupyter Notebook is just an entry in the cells list. One of the properties a cell has is metadata. Jupyter Notebook clients, like the built-in notebook viewer in VSCode / Cursor, have a feature which leverage this metadata property to add tags.
In VSCode, we can do this by opening the command pallete and running the \"Add Cell Tag\" command.
The JSON for a cell with source code: print(\"hello world\") tagged with example-tag looks like this:
{
\"cell_type\": \"code\",
\"execution_count\": null,
\"id\": \"16bb7324\",
\"metadata\": {
\"tags\": [
\"example-tag\"
]
},
\"outputs\": [],
\"source\": [
\"print(\\\"hello world\\\")\"
]
}
Using tags allows us to reference a specific cell very easily, in a way both I and my LLM understand.
For example I can prompt my LLM with:
In @notebooks/test.ipynb, in the cell tagged example-tag, change the print statement to all caps
and it would very easily make the change without reading irrelevant cells and bloating context.
In this case, I could just as easily reference the cell by it's index like:
In @notebooks/test.ipynb, In the first cell, change the print statement to all caps
but as a notebook grows to dozens of cells, this becomes untenable.
Using cell tags scales.
Here's where my mind was really blown – running cells in agentic loops.
Say I wanted my LLM not only to make the change to print \"hello world\" in all caps, I want it to verify that after making the change and running the cell, the output of that cell indeed prints \"HELLO WORLD\".
Again referencing the cell tag, I can ask my LLM:
\"In @notebooks/test.ipynb in the cell tagged example-tag, change the print statement to all caps, run the cell, and verify that the printed text is in all caps\"
In this case, your agent should use a tool maintained by the Jupyter Development Team called nbconvert, which, among other things, enables Jupyter notebook execution from the command line.
For our test.ipynb notebook, the command will look like:
jupyter nbconvert --to notebook --execute test.ipynb
This will populate the value of the \"outputs\" key of your cells. The agent can then look up the cell in question by its tag, read the \"outputs\" value, and verify the output using its great natural language ability or by writing an ad-hoc script.
See an example session here walking through what was described above. The agent makes a code change by modifying the \"source\" value of a tagged cell, uses jupyter nbconvert to execute the notebook, and reads the \"outputs\" value of the cell to check that our all caps string was printed to stdout.
I've consistently been using worktrees for the past two years.
But I recently learned git clone --bare is suboptimal – there's a better way to clone a repo for worktree usage that doesn't require cluttering up your repo root or creating extra branches for the worktree workflow.
Here's the fastest, cleanest way I've found to clone a repo for usage with worktress. Works every time, without needing to run git gc all the time to clean up storage bloat.
I use it in zsh, but it should work in bash. (Fish users, sorry, I have no clue)
function git-empty-clone() {
# Useful for cloning a repo suitable for use with git worktrees
# Taken from this stack overflow answer:
# https://stackoverflow.com/questions/54367011/git-bare-repositories-worktrees-and-tracking-branches
if [ $# -lt 2 ]; then
echo \"Usage: git-empty-clone <repo-url> <target-dir>\"
return 1
fi
local repo_url=\"$1\"
local target_dir=\"$2\"
git clone \"$repo_url\" \"$target_dir\" || return 1
cd \"$target_dir\" || return 1
# Create an empty root commit
local empty_tree
empty_tree=$(git hash-object -t tree /dev/null)
local empty_commit
empty_commit=$(echo | git commit-tree \"$empty_tree\")
# Checkout the empty commit in detached HEAD
git checkout \"$empty_commit\"
echo \"Use git worktree add <branch_name> to start using worktrees.\"
}
Example usage to clone my vim configs repo, then add a worktree for the dev branch:
git-empty-clone git@github.com:whoislewys/vim_runtime.git vim_runtime.git
git worktree add dev
Read the thread to learn why it works and uncover the mystery of Git's empty tree.
The .worktreeinclude approach is perfect, and scales nicely to monorepos.
Before coming across it, I was using a post-worktree-create git hook that conflicted with husky pre-commit hooks, and I ended up having to invoke it manually so as not to bloat the pre commit hooks for team members not using worktrees. A mess.
", "url": "https://whoislewys.com/blog/2025/10/09/a-better-way-to-worktree", "title": "A Better Way to Worktree", "summary": "I've consistently been using worktrees for the past two years.", "date_modified": "2025-10-09T00:00:00.000Z", "author": { "name": "Luis Gomez", "url": "https://github.com/whoislewys" }, "tags": [ "git", "cli" ] }, { "id": "https://whoislewys.com/blog/2024/10/18/why-i-doxxed", "content_html": "TODO: write support for pi, my preferred agent harness, to use
.worktreeinclude
When I started working on crypto projects, I chose to work under a pseudonym. Here's why, and why I ultimately chose to reveal my identity.
The less they know, the better.Stay anon. https://t.co/IvEDWtPQhd
— BowTiedBull - Read Pinned Tweet or NGMI (@BowTiedBull) May 7, 2023
Targeted robberies, SIM swaps, and harassment happen. Reducing links back to you reduce your risk.
The application developers I looked up to the most in the space were anons: TzTokChad, 0xSami, Banteg, Zeus, Fully... I could go on.
Their market knowledge and technical chops spoke for themselves. There was no need to lean on the reputation of their \"previous life\".
I wanted to be like that.
I wanted to become so good I couldn't be ignored, regardless of my name, race, or history.
Also, being anon just looked cool imo \"¯\\_(ツ)_/¯\"
It felt overly risky betting all of my professional reputation on such a niche field.
Using a pseudonym would allow me to build legitimacy in the space without associating my real identity with the \"crypto dev\" label - one widely associated with vaporware and scams.
Meanwhile, I could maintain my existing reputation by keeping up my traditional software skills and relationships.
In summary, I liked the optionality of pseudonymity. If crypto became more socially acceptable, I became valuable enough in the niche, or I got tired of the mask, I could always exercise my option to dox.
And that's what I chose to do.
In the end, the opportunity cost of being anon bothered me. Also, I realized I was overestimating my personal robbery risk.
You have to build credibility, and you\nhave to do it under your own name as much as possible.
- Naval Ravikant
While being a high-integrity anon is respectable and profitable, it can have a high opportunity cost.
If there is a large overlap between the goals of your pseudonym and your true identity, you're missing out on the compounding returns of accountability. And if you know anything about managing money, you know compounding returns are the juiciest.
Being anon is also a lot of upkeep. Keeping accounts straight across multiple SIM cards, keys, and other opsec drudgery started cutting into my personal and professional life.
That said, pseudonymity makes sense in a lot of cases.
If you can grow or have fun by expressing yourself in a way that clashes with societal norms, try it as an anon! Making a throwaway account to ask Reddit bondage questions as a Fortune 500 CEO is a wise choice. Calling yourself BasedBeffJezos and inspiring a wave of techno-optimism is also a great cause.
Or, if pseudonymity is your only hope for accomplishing a net good for humanity that the authorities would consider \"subversive\", tighten up your opsec and get to it (RIP TornadoCash).
But for my case, where my main quest is to accumulate wealth, using a pseudonym was doing more harm than good.
Another reason I doxed is because I overestimated my personal robbery risk.
The things that put me at ease are my travel frequency and high personal security standards.
I travel frequently enough that pinning my location down remains tricky. Obviously, not everyone has this luxury. What else can you do?
Well, don't walk around with your trezor in your pocket, its code written on your forehead while live streaming your location.
On top of that, you can keep the expected returns to a violent attack on you low by making your assets difficult to retrieve. Copytrade Vitalik - store crypto assets behind a combination of hardware wallets and multisigs. Keep more traditional accounts safe with multi-factor auth methods like Yubikey and OTP codes. Don't use text messages for 2FA, or you'll get rekt by a low-tier SIM swap attack.
May Terrance Andrew's Research Center live on in our hearts and git histories.
- 0xTARC Luis
TLDR; You can simply use: useQuery<SerializeFrom<typeof loader>>
The blog post was meant to pile on the bandwagon of laughing at how ridiculous TypeScript types can get.
However, after playing with Shopify Oxygen, I recently learned about the (undocumented) Remix type SerializeFrom.
Instead of littering your code with the ridiculous thing I derive in the blog post:\nuseQuery<Awaited<ReturnType<Awaited<ReturnType<typeof loader>>['json']>>>
Is it bad that this common use case of wanting to re-use the data type returned from your loaders in your client is undocumented and hard to find (as of May 20 2024)? Yes of course. But at least it's there!
This also goes to show the utility of TypeScript's type system. Usually when you see a ridiculous looking type, there's a way to clean it up for downstream consumers.
Leaving the original blog post below to document my madness for posterity.
Using TanStack Query to fetch a resource route in your Remix app?
Want a strongly typed response?
lmaooo
JK here you go:
useQuery<Awaited<ReturnType<Awaited<ReturnType<typeof loader>>['json']>>>
Yay TypeScript!
Want a full example? Surely you're intrigued by now...
import { loader } from '~/routes/your_resource_route'
const {
data,
isLoading,
error,
} = useQuery<Awaited<ReturnType<Awaited<ReturnType<typeof loader>>['json']>>>(
['your_loader_name', queryString],
() =>
fetch(`${window.location.origin}/your_resource_route`).then(
(res) => res.json(),
),
{
enabled: typeof window === 'object', // fetch only on client
},
)
To fetch and get the response in JSON format in one line, you do:
await(await fetch('')).json()
And loaders are pretty much just fetch under the hood.
So, to infer the loader's response, we use Awaited<ReturnType<typeof loader>
To infer the .json() part, we use ReturnType<...>['json']>
And finally, because for some reason the json method is async, we wrap everything in a final Awaited<...>
<Awaited<ReturnType<Awaited<ReturnType<typeof loader>>['json']>>>(
Yay!
The enabled: typeof window === 'object' argument sidesteps all the complex initialData, hydration, and dehydration stuff mentioned in the TanStack Query docs.
Note that it will limit this fetch to the client, so this pattern should only be used for components with complex fetching logic that don't need to render right away.
Why not use the the amazing defer API?
My use case is a complex chart with lots of data to paginate through. Previously I was fetching it in my loader and the page took 4 seconds to paint. Now the page paints in 0.5 seconds.
Also, the pagination makes re-using the page loader a bad idea - if I request a new chunk of data for the chart from the loader (ex. ?chart_page=2), the entire loader would run again, re-rendering the entire route & its children. All I want is the chart to re-render with a new chunk of data, full-stack component style.
Having a working knowledge of Solidity for a crypto developer in the EVM ecosystem is critical. You can be much more productive when you have the ability to dive into the source yourself. And diving into Solidity source code becomes much easier when you have deep integration in your dev environment of choice. So of course as a neck-bearded Neovim user, I wanted to share how I set this up!
As of Solidity 0.8.11, the Solidity team has added LSP support to solc, “which means every IDE that has Language Server support will support Solidity out of the box.”
https://blog.soliditylang.org/2021/12/20/solidity-0.8.11-release-announcement/
This makes it easy for us Vim / Neovim users to get intelligent code actions (autocomplete, go to definition, batch renames, etc.) when reading and writing Solidity!
So here’s my guide on the easiest, fastest way to get Solidity support in your Vim / Neovim environment:
What is asdf? asdf is a meta version manager - it gives you a common interface for installing and switching between multiple versions of multiple languages. I recommend asdf because it makes it easy to install and switch between versions of solc, node (for Typescript!), and other languages. The only language I’ve found it doesn’t play nice with is Rust. Good thing Rustaceans have rustup!
Now add the official Solidity plugin to asdf and use it install your desired version of solc by running:
asdf plugin add solidity [<https://github.com/diegodorado/asdf-solidity.git>](<https://github.com/diegodorado/asdf-solidity.git>)
asdf install solidity 0.8.9
asdf global solidity 0.8.9
You can substitute 0.8.9 for latest, or any other specific version of solc.
Run solc --version to ensure it all worked.
asdf install solidity 0.8.12 to install, and asdf global solidity 0.8.17 to make it available in your pathNow install the coc-solidity plugin to get the Language Server actions into Vim by opening Vim / Neovim and running:
:CocInstall coc-solidity
All done! Try it out by opening a .sol file, going to a variable usage, and trying the “go to definition” action. If you followed the Coc.nvim example configuration, you can do this with gd in normal mode.
That was 5 easy steps to get Solidity LSP support in Vim and/or Neovim. Have fun, and please reach out on Twitter if you have questions or suggestions for future posts @0xTARC.
Bye for now,
I found an interesting package when I was looking into the rainbowkit source: Lavamoat.
What it do?
Quoting @metamask’s Lavamoat blog here,
“JavaScript is the most popular language for developers by far and is also very prone to supply chain attacks. This vulnerability timeline from 2017 shows a solid half of the attacks originating from npm…”
Basically, a simple npm install can rek you and your users in all sorts of ways! From an install script stealing your private keys (hope you use a separate wallet for development), to injecting hidden functionality into your app, supply chain attacks can hit from multiple angles.
Thankfully, @kumavis_ rode his terminal to the future and brought us back Lavamoat.
There are a few tools offered by LavaMoat in their GitHub repo to avoid supply chain attacks at different stages, but today I’m going to show off @lavamoat/allow-scripts since it’s an easy starting point and prevents the most common supply chain attack vector: malicious install scripts.
And, it’s easy to use.
Here’s how:
Install the package (ironic I know, but stay with me here) with:
yarn add -D @lavamoat/allow-scripts
OR
npm i -D @lavamoat/allow-scripts
Setup, which adds a .yarnrc or .npmrc and the @lavamoat/preinstall-always-fail package to prevent preinstall scripts from running automatically, by doing:
yarn allow-scripts setup
OR
npx --no-install allow-scripts setup
Configure your package.json to run necessary scripts by running:
yarn allow-scripts auto
And editing the new lavamoat section in your package.json.
You can find more details and up-to-date instructions on their npm page!
Please subscribe to stay up-to-date with the latest and greatest in web3 frontends.
And please leave a comment / DM me on Twitter letting me know what you’d like to hear about next! Tutorials on building dApps? UI/UX critiques / refactors? Open to anything.
Thanks!
As a web3 developer for a full year, I've read a TON of open source dApp front-end code. One thing that always concerned me was the widespread usage of Next.js.
Next.js is a server side rendering (SSR) framework for React. There's a few problems with this:
There’s no need for the additional complexity of Next in dApps.
Do you know what Next.js consists of? It’s 38 MB, ser. I can damn near guarantee you don’t need all of that for your dApp.
Decentralized server-side compute is far from a solved challenge, so most dApps just get hosted on Vercel forever.
This makes frontends built with Next the weakest link in a full-stack dApp.
“But Taaaarc, frontends don’t really matter, users can interact with the contracts themselves, or fork the frontend and run it themselves!1!1!!
cast or ethers.js?Just take a look at the “Unsupported Features” section of the Next.js docs on static HTML exports.
A particularly damning subset:
There are some other gotchas too that most Next users probably don’t even know about. RTFM! :3
First-class support for static builds is important because hosting a static build on a decentralized storage network like Filecoin is the best way to host a decentralized frontend.
People who prefer Vue, for example, can feel excluded from dApp development due to the widespread usage of Next.
So anon, don’t start your next hackathon project or dApp prototype from a Next.js template.
“Ok Next.js bad. Wat do Tarc?”
Use Vite instead.
What is Vite?
It’s a blazingly-fast build tool for JavaScript. With Vite, you get:
Did I mention Vercel has first-class support for Vite? So you can still have fast & convenient preview builds through Vercel.
Sold yet? Let’s learn how to create a new React-based dApp with Vite in ONE easy command.
Let’s use the create-wagmi cli to scaffold a new dApp. It’s just one simple command:
npm init wagmi -- --template vite-react-rainbowkit
Follow the instructions from the create-wagmi cli to start the app. You should see something like this:\nwagmi + rainbow <3
It’s important to remember that part of our jobs as software engineers is to choose the right tool for the job.
I have yet to see a case where Next.js is the right tool for building a dApp, yet it’s used EVERYWHERE in web3, even down to the dev tool documentation.
It’s time to stop. You need to be Vite-maxxing.
Thanks for reading as always and remember to subscribe for more actionable advice on Web3 frontends, UI, and UX!
Bye bye
It’s no mystery that interacting with blockchains is still hard compared to the buttery-slickness of today’s FinTech.
Think about the difference between buying Minecraft on the App Store with PayPal, and buying an NFT on OpenSea with MetaMask.
This is the gap we need to fill.
And the novice users aren’t the only ones struggling.
Even “crypto natives” struggle with things like
We can do better 🙂
But how? First, let’s define what a wallet is. In UX-speak, what are the “affordances” of a wallet?
When I say “wallet” I’m not thinking about a hunk of metal engraved with 24 magic words. I’m thinking along the lines of MetaMask or Coinbase Wallet - something you would use every day for financial transactions. In general, this wallet should let you do the following:
Most people use Trust Wallet or MetaMask for these abilities.
Some use hardware wallets like Ledgers, which can add even more complexity (worth it!).
And multi-sigs signers? Forget it. Advanced users only, and we quadruple check every step of the way.
It can be so much better. Here are some tools and ideas to improve wallet UX:
Gnosis Safe Modules. Zodiac has a ton of great ones.
Zodiac, “The expansion pack for DAOs”, provides amazing extensions for your multi-sig wallets like Gnosis Safe. They provide Modules, like Exit for DAOs built on Gnosis Safe to allow their members to “rage quit”, or Reality to do things like execute on-chain transactions based on the results of a Snapshot vote. There’s also Modifiers, which includes Roles for roles-based access control and Delay for time-delayed contract execution, and Guards, like Scope for whitelisting specific contract interactions.
delegate.cash, allows for safe delegation of certain transactions involving cold wallet assets from a hot wallet.
MetaMask Flask and Snaps.
MetaMask Flask is the “canary” build of MetaMask for developers that lets you install Snaps. Snaps are a really exciting feature that let you build and install extensions for MetaMask. Some of my favorite snaps:
Creates a “session” where transactions get auto-signed until the session ends. Similar to how you can sign into a bank account, do multiple transactions, then sign out. This is amazing for blockchain gaming.
Multi-Chain snaps, allowing you to interact with non-EVM blockchains through MetaMask.
BTCSnap, FilSnap, ArSnap, metamask-snap-polkadot , etc.
The free human-readable usernames make sending and receiving money between friends a breeze. We shouldn’t need to keep a list of our friends’ public keys. An integrated fiat on-ramp too? Nice. I recommend this one, paired with a Trezor or Ledger, to people new to crypto.
Please build this!
Clunky? Well the Snap Store aggregates all Snaps, with ratings. You can even integrate payments, allowing developers to monetize MetaMask plugins.
Hopefully you learned something, anon! Thanks for reading and subscribe for more actionable advice on Crypto UI, UX, and frontend development.
Peace :3
DeveloperDao’s MetaMask Snap tutorial
", "url": "https://whoislewys.com/blog/crypto-ux-sucks-wallet-edition", "title": "Crypto UX Sucks: Wallet Edition", "summary": "Crypto Wallet UX Sucks", "date_modified": "2023-01-21T00:00:00.000Z", "author": { "name": "0xTARC", "url": "https://github.com/whoislewys" }, "tags": [ "blog" ] }, { "id": "https://whoislewys.com/blog/typechain-ethers-sdk-guide", "content_html": "Have you ever
bytes32 should be in TypeScript?Me too, anon.
What if we could have ONE place that keeps track of all of our addresses, ABIs, AND ALSO makes interacting with our contracts 10x easier? We can with TypeChain! Let’s see how in 5 simple(ish) steps.
We can make our lives WAY easier in 5 simple steps:
If you’d like to follow along, the smart contract repo I’ll be using for this example is hardhat-smartcontract-lottery-fcc from Patrick Collins’ excellent, free Full-Stack Web3 Development course.
Check your smart contract repo for committed ABIs. In a default Hardhat project, it’s the artifacts/ directory. With Foundry, the out/ directory. Do some searching, and hopefully you’ll find those .json files somewhere.
If the ABIs aren’t there, compile the contracts yourself and commit the ABIs. Or ask your smart contract devs to do it (with an uwu and pwetty pwease on top).
For the hardhat-smartcontract-lottery-fcc repo, I did the following:
.gitignore and removed artifacts/npx hardhat compile to generate ABIs in the /artifacts directoryIt’s very important to have up to date ABIs committed. Ideally this process happens in CI so it’s harder for them to go missing or get out of date.
Now that we have our ABIs tidied up and TypeChain installed, we’re done with the smart contracts. Now we just need a basic TypeScript repo. Something like this will do the trick. I recommend using that as a reference.
Check the docs or ask your smart contract devs where to find the addresses for the latest deployed contracts.
We’re going to put these in our SDK in a new file: src/addresses.ts.
Using Raffle.sol as an example:
export const Addresses: Record<number, any> = {
31337: {
Raffle: '0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9',
},
5: {
Raffle: '0xBCAd87730Fa664ee16b6a599A3a5A94C020e1255',
},
}
So it maps the chain ID → Contract Name → Address.
Here I show two chains: 31337 is chain ID of the Hardhat network for local development, and 5 is Goerli, ‘cause I’m too poor for Mainnet. Hopefully it’s obvious how this can expand to support any number of EVM-ish chains.
If we were really fancy, we’d use a continuous integration process to update the ABIs, deploy the contracts, save deployed addresses, and PR them to this repo on every push to main. But that’s a whole other post.
“Wooow Tarc, you taught us how to commit files - groundbreaking. And you know what ‘CI’ means… cute. What about all that ‘making smart contract interactions easy’ stuff? I’m outta here...”
Enter TypeChain.
It’s easy to use:
Let’s go! We’re 80% there!
In your SDK repo, install your smart contract repo as a package with:
npm i git@github.com:0xTARC/hardhat-smartcontract-lottery-fcc.git#bb8b04d
IMPORTANT: #bb8b04d is the latest commit hash of the smart contract repo. You want to make sure to include this when using git dependencies to avoid accidentally introducing breaking changes in your SDK.
When you’re ready for a new SDK version, you’ll re-install your smart contract repo with a new commit hash, bump the version in package.json, and re-release.
Now, we need to add some scripts to run typechain and generate code for us. Open the package.json of the SDK repo, go to the scripts section, and make it look something like this:
\"scripts\": {
\"build\": \"rm -rf ./dist && tsc\",
\"gen-hardhat-smartcontract-lottery-bindings\": \"find ./node_modules/hardhat-smartcontract-lottery/artifacts -name '*.json' ! -name '*dbg.json' ! -path './node_modules/hardhat-smartcontract-lottery/artifacts/build-info/*' | xargs npx typechain --target ethers-v5 --out-dir src/types\",
\"prebuild\": \"npm run gen-hardhat-smartcontract-lottery-bindings\"
},
Finally, run npm run build. This will run the prebuild script, which runs the gen-hardhat-smartcontract-lottery-bindings script, which uses scary shell commands to run typechain on our committed ABIs from step 1.
That’s it! Your shiny new SDK is sitting in the /dist folder. You can use npm link to test locally or npm publish to test in prod (shoutouts Andre 🙏).
If you have a complex system of smart contracts living in multiple repos, install those repos as NPM dependencies too, add additional
scriptsto yourpackage.json(e.g.\"gen-foo-bindings\": \"<typechain command>\") and make sure to add those new scripts them in yourprebuildscript (e.g.\"npm run gen-hardhat-smartcontract-lottery-bindings && npm run gen-foo-bindings”).
You might wonder why installing a smart contract repo as an npm package is even possible, and if it would ever break. It’s admittedly a hack, and not future proof, but I it would work for 90% of cases. This is because our smart contract repo has a
package.json, pretty much the only requirement for a Node package. This will obviously work for every Hardhat project because Hardhat is just a Node package. BUT it also works with most other projects due to the common practice of using Node packages to lint and format Solidity. For example, PRB’s Foundry template, and Transmissions11’s solmate. So If your smart contract repo doesn’t have apackage.jsonand you’re against adding one, you can’t follow the rest of this tutorial word-for-word, but the principles are the same (usecurlto download your repo instead, modify type generation command, etc). Anyways, let’s move on assuming you cannpm installyour smart contract repo.
I put together a small example repo to show how you would use this SDK’s TypeChain bindings if you were to writing some TypeScript to interact with the Raffle contract.
Here’s some excerpts, with some links to the full code:
Instantiating the Contract (assuming you have a valid signer and/or provider):
import { Addresses, Rafflefactory } from '@0xtarc/example-sdk';\nconst RaffleAddress = Addresses[chainId].Raffle;\nconst Raffle = Rafflefactory.connect(RaffleAddress, signer ?? provider);
Calling getEntranceFee (Performing a read call):
const entranceFee = await Raffle.getEntranceFee();
Calling enterRaffle (Performing a write call):
const enterRaffle = () => {\nRaffle.enterRaffle({\nvalue: entranceFee,\n});\n};
Listening for RaffleEnter events and logging new entrant addresses:
const RaffleEnter = () => {\nRaffle.on(Raffle.filters.RaffleEnter(), (entrant) => {\nconsole.log('New entrant: ', entrant);\n});\n};
You can see the full repo here. App.tsx is probably the file you want. Play around and see how good the autocomplete is!
This repo is based off the Rainbowkit example app
with-vite, the best way to start a new dApp. That’s another post too - stay tuned.
Hopefully you learned how to create a robust, flexible, envy-inducing SDK with TypeChain and Ethers! TypeChain is awesome and the more you work with it, the more you’ll like it.
Thanks for reading! Please subscribe to keep leveling up your Web3.0 crypto blockchain frontend skillz!
Peace ✌️
Patrick Collins’ FREE “Full Stack Web3 Development” course doesn’t need more shills, but I’ll shill it anyway. It’s awesome for going from Blockchain Basics → Jr. Full-Stack Web3 dev. Here’s the Smart Contract Lottery portion:
", "url": "https://whoislewys.com/blog/typechain-ethers-sdk-guide", "title": "The Ultimate Guide to Stress-Free, Multichain Smart Contract Calls in TypeScript: How to Build an SDK with TypeChain and Ethers", "summary": "The Pain of Calling Smart Contracts from the Frontend", "date_modified": "2023-01-15T00:00:00.000Z", "author": { "name": "0xTARC", "url": "https://github.com/whoislewys" }, "tags": [ "blog" ] }, { "id": "https://whoislewys.com/blog/defi-health-factor-slider-react-component", "content_html": "So you need your users to choose from a range of values while giving them a hint about how risky their choice is.
Maybe you reach for Material UI, install 6MB of potentially dangerous dependencies for a single, hard to customize component.
Maybe you're bought-in to the headless component wave and are in the middle of migrating to your own design system (my case!).
Or maybe you just want a great slider with minimal dependencies.
No matter your situation, this one's for you, anon.
So, here's the Slider. It’s based on a Slider I built for the canonical FIAT protocol frontend.
It’s built with Typescript, React, and Radix UI. The Slider is styled with Stitches, though with a little effort you can swap for CSS Modules or another styling solution.\n![](\"https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6609b3f8-b240-4a44-926e-8fa1210c2fee_1618x938.gif\")
\nEdit on CodeSandbox
Details of note:
color variant with Stitches. Variants are a powerful way to express a lot of style without a lot of code. Plus, a ton of devs are used to using variants thanks to MUI (color='primary’, anybody?). If you're interested, read more on variants here.[data-inverted], is added to SliderRoot. This is then used as a selector to reverse the gradient. This is in line with the intuitive Radix component APIs, and makes it easy to port the styling of this component over to CSS Modules, or another styling solution. (CSS Modules ftw, I just happened to have Stitches installed because we're migrating off Next UI.)![\"Slider](\"https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb4d4231d-7dd7-4b82-80d2-a9dfbe312d8f_1012x760.gif\")
Good types = good autocompletion = happy devs\nHad to do some Typescript-fu to get that to work, just look at how wonky the SliderProps interface is. But from my experience with design systems, wonky types are often the case when creating components that will have good autocompletion for devs.
Tried for a few mins to replicate that choice Developer eXperience™️ with vanilla-extract but couldn't quite get it. If you can, please message me and I'll add that example to the CodeSandbox & repo.
Thanks for reading! :3
What follows is an exact repost/mirror of Steve Yegge's 2012 post \"Notes from the Mystery Machine Bus\" from Google+ on the software risk aversion axis. It's not even accessible from the WaybackMachine. Enjoy.
I've spent the past eight years (starting back in June 2004) writing elaborate rants about a bunch of vaguely related software engineering issues.
I was doing all that ranting because I've been genuinely perplexed by a set of \"bizarre\" world-views held dear by -- as far as I can tell -- about half of all programmers I encounter, whether online or in person.
Last week, after nearly a decade of hurling myself against this problem, I've finally figured it out. I know exactly what's been bothering me.
In today's essay I'm going to present you with a new conceptual framework for thinking about software engineering. This set of ideas I present will be completely obvious to you. You will probably slap yourself for not having thought of it yourself. Or you might slap the person next to you. In fact you probably have thought of it yourself, because it is so blindingly obvious.
But in my thirty-odd years as a programmer I'm pretty sure this way of thinking about things, if it already existed, has never been mainstream. That assertion is backed by what has to be at least ten Google searches that turned up nothing. So I'm pretty confident.
I'm going to make it mainstream, right now. Watch!
And I suspect this conceptual framework I'm giving you will immediately become, and forever remain, one of the most important tools in your toolkit for talking with -- and about -- other programmers.
The punch line, a.k.a. TL;DR
I won't keep you in suspense. Here is the thesis of this looooong essay. It is the root cause that motivated over half of my ranting all these years, starting at Amazon and continuing here at Google.
(Note: I Do Not Speak For My Employer. This should be patently obvious. When employers want someone to speak for them, they hire a person like the Mouth of Sauron, to make absolutely sure everyone knows they are speaking for the Employer.)
(Note: Technically, you could stop reading right here and be at pretty much 90% comprehension. In case you care.)
The notions of \"conservative\" and \"liberal\" on this political axis are specialized to software engineering. But they exhibit some strong similarities to their counterparts in real-world politics.
Everyone in the software industry who does stuff related to programming computers falls somewhere fairly precise on this political spectrum, whether they realize it or not.
Put another way, YOU are either a liberal or a conservative software engineer. You may be more of a centrist, or maybe an extremist, but you fall somewhere on that left/right spectrum.
Just as in real-world politics, software conservatism and liberalism are radically different world views. Make no mistake: they are at odds. They have opposing value systems, priorities, core beliefs and motivations. These value systems clash at design time, at implementation time, at diagnostic time, at recovery time. They get along like green eggs and ham.
I think it is important for us to recognize and understand the conservative/liberal distinction in our industry. It probably won't help us agree on anything, pretty much by definition. Any particular issue only makes it onto the political axis if there is a fundamental, irreconcilable difference of opinion about it. Programmers probably won't -- or maybe even can't -- change their core value systems.
But the political-axis framework gives us a familiar set of ideas and terms for identifying areas of fundamental disagreement. This can lead to faster problem resolution. Being able to identify something quickly as a well-defined political issue means we can stop wasting time trying to convince the other side to change their minds, and instead move directly into the resolution phase, which (just as in politics) generally boils down to negotiation and compromise. Or, you know, Watergate.
Are real politics and software politics correlated?
Does being a political conservative make you more likely to be a software conservative? I think all we have are hunches and intuition at this point. We'd need studies for an answer with any scientific basis.
But my hunch tells me that they are only loosely correlated, if at all. I suspect that the four possible combinations of political left/right and software left/right each contain around a quarter of the programming population, plus or minus maybe 8%. But that's a totally nonscientific hunch, and I'd love to see the real numbers.
The reason I suspect they're only loosely correlated is that I think a person's software-political views are largely formed by two forces:
Their personal early/formative experiences as a programmer -- in particular, what kinds of decisions either burned them or made them wildly successful.
The software-political core values of their teachers, professors, mentors, role models, and coworkers.
But there are some factors that could increase correlation between political orientation and software orientation. One is geographic/regional influence -- for instance, big tech universities being located primarily in red or blue states. Another might be basic personality traits that may incline people toward traditionally liberal or conservative value systems.
Regardless of how correlated they are, there are going to be a lot of programmers out there who are mildly shocked to find that they are political conservatives but software liberals, or vice-versa.
Well, yeah. Mapping something as nuanced as political viewpoints onto a one-dimensional axis is a well-known gross oversimplification. For instance, some U.S. voters may self-identify as fiscally conservative but socially liberal, and as a result may wind up not having a strong tie to either majority political party. These \"swing votes\" tend to be the reason that political moderates more often wind up winning elections, even though studies have shown that people tend to assign more credibility to extreme opinions.
The influence of centrists or moderates is just as important in the software engineering world, and we'll explore it more in a bit.
But having a single axis still allows for a wide distribution on both the right and the left. As in real politics, a moderately right-leaning viewpoint may seem dishearteningly liberal to someone who lies even further toward the extreme right. I will illustrate with some well-known real-world programming language scenarios in the examples section.
All in all, despite its oversimplifications, I think 2-party model is a good starting point for educating people about the politics of software. It'll be a good frame of reference for refining the model down the road, although that's out of scope for this essay.
If you ignore specific real-world issues, and just focus on the underlying traits and values of real-world conservatives and liberals, it boils down to just a few themes. I'll argue that those underlying themes are also the basis for software politics.
It's easiest to talk first about conservatives, and then define liberals in terms of what conservatives are not. This is because conservatives tend to have a unified and easily-articulated value system, whereas liberals tend to be more weakly organized and band together mostly as a reaction to conservatism. This applies to both real-world and software-world politics.
So we'll start with an operational definition of conservatism, from Jost et al.:
\"We regard political conservatism as an ideological belief system that is significantly (but not completely) related to motivational concerns having to do with the psychological management of uncertainty and fear.\"
This theory is explored, re-tested and affirmed in a 2008 study from Garney et. al, \"The Secret Lives of Liberals and Conservatives: Personality Profiles, Interaction Styles, and the Things They Leave Behind\".
I hope you'll agree that this definition is minimally controversial. After all, the adjective \"conservative\" is more or less synonymous with caution and risk aversion. Financial conservatism is frequently (and intuitively) associated with age and with wealth. Companies tend to grow more conservative with age as they weather the storms of lawsuits, technical fiascoes, dramatic swings of public opinion, and economic downturns. We even have fables about ants and grasshoppers to teach us about conserving food for the upcoming winter.
Conservatism, at its heart, is really about risk management.
Similarly, liberal views are often associated with youth, with idealism, with naivete. In the corporate world, we think of startups as being prototypically liberal -- in part because they're setting out to change the world in some way (and liberalism is traditionally associated with change), and in part because they have to go all-out in order to hit their scheduled funding milestones, which can justify cutting corners on software safety.
Liberalism doesn't lend itself quite as conveniently to a primary root motivation. But for our purposes we can think of it as a belief system that is motivated by the desire above all else to effect change. In corporate terms, as we observed, it's about changing the world. In software terms, liberalism aims to maximize the speed of feature development, while simultaneously maximizing the flexibility of the systems being built, so that feature development never needs to slow down or be compromised.
To be sure, conservatives think that's what they're maximizing too. But their approach is... well, conservative. Flexibility and productivity are still motivators, but they are not the primary motivators. Safety always trumps other considerations, and performance also tends to rank very highly in the software-conservative's value system.
The crux of the disagreement between liberals and conservatives in the software world is this: how much focus should you put on safety? Not just compile-time type-safety, but also broader kinds of \"idiot-proofing\" for systems spanning more than one machine.
Let's characterize this core disagreement with some example statements that would be rated with much higher importance by conservatives than by liberals:
Software should aim to be bug free before it launches. (Banner claim: \"Debugging Sucks!\") Make sure your types and interfaces are all modeled, your tests are all written, and your system is fully specified before you launch. Or else be prepared for the worst!
Programmers should be protected from errors. Many language features are inherently error-prone and dangerous, and should be disallowed for all the code we write. We can get by without these features, and our code will be that much safer.
Programmers have difficulty learning new syntax. We should limit the number of languages used at our company, so that nobody has to learn a new syntax when a system goes down in the middle of the night on Christmas Eve. And we should never permit features that allow defining new syntax, nor changing the semantics of existing syntax. (Common examples: no operator overloading, and NO metaprogramming!)
Production code must be safety-checked by a compiler. Any code that cannot be statically checked should in general be avoided. In specific cases where it is strictly necessary, uses of it must be approved by a central committee. (Examples: eval, dynamic invocation, RTTI).
Data stores must adhere to a well-defined, published schema. Relational databases must be in third-normal form and adhere to a UML or equivalent schema definition. XML should have a DTD. NoSQL databases and name/value stores -- both of which should be avoided in general -- must have a separate physical schema that defines all permissible keys and their corresponding value types.
Public interfaces should be rigorously modeled. Data should never be stored in strings or untyped collections. All input and output entities should be thorougly and explicitly specified via statically-checkable, ideally object-oriented models.
Production systems should never have dangerous or risky back-doors. It should never be possible to connect to a live production system via a debugger, telnet shell, nor any other interface that allows the developer to manipulate the runtime operation of the code or data. The only ports into a production system should be read-only monitoring channels.
If there is ANY doubt as to the safety of a component, it cannot be allowed in production -- no matter how teams may cry and wail that they need it to make forward progress. (I'm talkin' to you, FUSE).
Fast is better than slow. Everyone hates slow code. Code should perform well. You should engineer all your code for optimum speed up front, right out of the box. Otherwise it might not be fast enough. Avoid using languages or DSLs or libraries that have a reputation for being slow. Even if they're fast enough for your current purposes, the requirements (or callers) could change, and suddenly the software would be too slow!
Of course these examples are meant to be illustrative rather than comprehensive. And obviously not all of them may necessarily be espoused by everyone who self-identifies as a conservative. But barring minor exceptions, they are all very common Conservative viewpoints.
To decide whether a system or a technology is liberal or conservative, just think of a bunch of independent \"safety checkboxes\". If a majority of them are checked, then the system is conservative as a whole. The more that are checked, the more conservative it is.
By way of comparison, here are the Liberal versions of the nine examples above. It might help to think of Scooby-Doo characters. The examples above are all from Fred, and the examples below are from Shaggy.
Bugs are not a big deal. They happen anyway, no matter how hard you try to prevent them, and somehow life goes on. Good debuggers are awesome pieces of technology, and stepping through your code gives you insights you can't get any other way. Debugging and diagnosing are difficult arts, and every programmer should be competent with them. The Christmas Eve Outage scenario never, ever happens in practice -- that's what code freeze is for. Bugs are not a big deal! (This belief really may be the key dividing philosophy between Conservative and Liberal philosophies.)
Programmers are only newbies for a little while. The steady state for a programmer's career is being smart, knowledgeable, creative, resourceful and experienced. Putting a bunch of rules in place to protect newbies from doing harm (or being harmed) is incorrectly optimizing for the transient case instead of the steady state.
Programmers figure stuff out amazingly fast when their jobs depend on it. People learn to read sheet music, braille, sign language, and all sorts of other semiotic frameworks. Hell, even gorillas can apparently do all that. Programmers don't need protection from syntax. They just need documentation and a little slack time to read up on it.
Succinctness is power. Code should be kept small. Period. If your static checking tools can't reason about the code, then the checking needs to be made smarter (e.g. by incorporating runtime data) rather than making the code dumber.
Rigid schemas limit flexibility and slow down development. Lightweight/partial/optional schemas are a better tradeoff. Moreover, the schema is often not well-understood until a lot of data is collected and a lot of use cases are thorougly exercised. So the schema should follow the code rather than precede it.
Public interfaces should above all else be simple, backward-compatible, and future-compatible. Rigorous modeling is just guessing at how the interface will need to evolve. It makes both forward- and backward-compatibility almost impossible, resulting in interface churn and customer unhappiness. Public interfaces should always do the simplest thing that could possibly work, and grow only as needed.
System flexibility can mean the difference between you getting the customer (or contract) vs. your competitor nabbing it instead. Security and safety risks in runtime production systems can be mitigated and controlled by logging, monitoring and auditing. There are plenty of existence-proofs of large systems with root-access backdoors and shells (e.g. RDBMS, online game servers) whose risk is controlled while still giving them world-class runtime flexibility.
Companies should take risks, embrace progress, and fiercely resist ossification. It doesn't matter how big your business is: it must grow or die. If you want to stay competitive, you have to make a conscious, often painful effort to take risks. Which means you'll need good recovery techniques for the inevitable disasters. But you need those even if you don't take risks. So take risks!
Premature optimization is the root of all evil. Get the code working first, focusing on correctness over performance, and on iterative prototyping over correctness. Only when your customers list latency as the top priority should you begin performing profiler-driven optimizations.
There you have it: Shaggy vs. Fred.
Just as in real-world politics, software liberals are viewed by conservatives as slovenly, undisciplined, naive, unprincipled, downright \"bad\" engineers. And liberals view conservatives as paranoid, fearmongering, self-defeating bureaucrats.
Once again, although I don't think the two camps will ever agree, I do think that mutual understanding of the underlying value systems may help the camps compromise.
Or at the very least, the conservative/liberal classification should help the two camps steer clear of each other. I think it is probably better to have a harmonious team of all-liberals or all-conservatives than a mixed team of constantly clashing ideologies. It's a lot like how vehicle-driving philosophies can differ regionally -- it's OK if everyone drives in some crazy way, as long as they ALL drive that way.
So Security Engineers are the most conservative, then, right?
Wrong! The reality here goes against intuition, and I mention it to show how poorly our intuition fares when it comes to interpolating software-political views from job descriptions.
Security engineers are deeply concerned with risk assessment and attack-surface management, so you might well guess that they would naturally tend towards software conservatism.
In practice, however, security engineers tend to be keenly aware of the tradeoffs between caution and progress, since they have to spend big parts of their day meeting with software teams who are often too busy (not to mention woefully underinformed) to spend a lot of time on security. Security engineers learn quickly to make well-informed and practical risk-management decisions, rather than conservatively always trying to be as safe as humanly possible.
Hence many security engineers are in fact software liberals. Some just swing that way.
Many of the security engineers at Google happen to be fans of Ruby -- both as an intrinsically secure language, and also as a nice, expressive language for writing auditing scripts and other security analysis tools. It wound up being fairly easy for me to get security sign-off for using Ruby in production for my first project at Google. In contrast, it was almost career-endingly difficult for me to get the same sign-off from our highly conservative systems programmers.
The reality is that almost any programming specialization is going to be divided into liberal and conservative camps. There are left- and right-wing versions of web developers, persistence engineers, protocol engineers, serving-system engineers, and most other sub-genres of programming.
I'm hard-pressed to think of any domains that tend to be mostly-liberal or mostly-conservative, with the sole exception of Site Reliability Engineering, which is sort of by definition a conservative role. For most of the other domains that initially came to mind -- for instance, data analysts, which at first I thought were uniformly liberal -- I've been able to think of specific teams or subdomains composed entirely of one or the other.
So in the end I think it comes down to personal preference. That's it. I don't think it's as domain-driven as much as it is personality-driven. Some people are just liberal, and some are just conservative, and that's how they are.
Before I continue, I will now issue the important disclaimer that I am a hardcore software liberal, bordering on (but not quite) being a liberal extremist.
This fact, by and large, is what has driven me to pen many of my most infamous and controversial rants, including (among others) \"Execution in the Kingdom of Nouns\", \"Portrait of a N00b\", \"Rhinos and Tigers\", \"Code's Worst Enemy\", \"Scheming is Believing\", \"Tour de Babel\", \"Dynamic Languages Strike Back\", \"Transformation\", \"Haskell Researchers Announce Discovery of Industry Programmer Who Gives a Shit\", and many others besides.
Sure, I sometimes write about other stuff. My in-progress \"Programmers View of the Universe\" series is motivated by an entirely different bugbear that has nothing at all to do with software politics. My \"Universal Design Pattern\" article is about a technique that transcends software politics and can be applied with equal effectiveness in either a conservative or a liberal manner. And I've written about video games, Japanese anime and random other stuff.
But I've come to understand that liberalism underlies a tremendous amount of my thinking. Even when I'm writing about management, I see that I am a liberal manager rather than a conservative one. And despite being both relatively old and relatively wealthy, I am also a political liberal -- both socially and fiscally.
Nevertheless I am going to try to represent both sides fairly and accurately in this essay. You know, mostly. I think it's most important for you to buy in to the left/right distinction itself. I think it's far less important whether you happen to agree with my particular side.
I'll consider this essay a success if a sufficient number of you agree that the liberal/conservative distinction is valid for software engineering, and that my particular left/right classification of various technologies and philosophies below seems intuitively reasonable. I'm fine with declaring success if we disagree on a few small details, as long as we agree on the overall picture.
Soooo... static typing enthusiasts are conservative, right?
Why yes. Yes, they are. Static typing is unquestionably one of the key dividing software-political issues of our time. And static typing is a hallmark of the conservative world-view.
In the conservative view, static typing (whether explicit or inferred) is taken on faith as an absolute necessity for modern software engineering. It is not something that one questions. It is a non-issue: a cornerstone of what constitutes the very definition of Acceptable Engineering Practice.
In the liberal's view, static typing is analogous to Security Theater. It exists solely to make people feel safe. People (and airports) have proven time and again that you can be just as statistically secure without it. But some people need it in order to feel \"safe enough\".
That's a pretty big difference of opinion -- one I'm sure you can relate to, regardless of how you feel about it.
I'm not going to try to defend my view here, since that's what all those old blog posts were for. If I haven't convinced you by now, then there's not much point in continuing to try. I respect your opinion. Well, now. And I hope you now have a little better understanding of mine.
I should, however, mention that there is an unrelated (i.e. politically-neutral) point on which both camps agree: namely, that static types yield better toolchain support. This is undeniably true today, and I have made it my life's work to ensure that it is not true tomorrow.
I have spent the last four years championing an initiative within Google called the \"Grok Project\", one that will at some point burst beyond our big walled garden and into your world. The project's sole purpose in life is to bring toolchain feature parity to all languages, all clients, all build systems, and all platforms.
(Some technical details follow; feel free to skip to the next section heading...)
My project is accomplishing this lofty and almost insanely ambitious goal through the (A) normative, language-neutral, cross-language definitions of, and (B) subsequent standardization of, several distinct parts of the toolchain: (I) compiler and interpreter Intermediate Representations and metadata, (II) editor-client-to-server protocols, (III) source code indexing, analysis and query languages, and (IV) fine-grained dependency specifications at the level of build systems, source files, and code symbols.
OK, that's not the whole picture. But it's well over half of it.
Grok is not what you would call a \"small\" project. I will be working on it for quite some time to come. The project has gone through several distinct lifecycle phases in its four years, from \"VC funding\" to \"acceptance\" to \"cautious enthusiasm\" to \"OMG all these internal and even external projects now depend critically on us.\" Our team has recently doubled in size, from six engineers to twelve. Every year -- every quarter -- we gain momentum, and our code index grows richer.
Grok is not a confidential project. But we have not yet talked openly about it, not much, not yet, because we don't want people to get over-excited prematurely. There is a lot of work and a lot of dogfooding left to do before we can start thinking about the process for opening it up.
For purposes of this essay, I'll assert that at some point in the next decade or so, static types will not be a prerequisite for world-class toolchain support.
I think people will still argue heatedly about type systems, and conservatives may never be able to agree amongst themselves as to which type system approach is best for modeling program behavior. But at least I will have helped that discussion be ONLY about representations. The quality of the corresponding developer tools should no longer be a factor in the discussions.
I'm going to go through and toss a bunch of random technologies, patterns, designs and disciplines each into one of six buckets: \"apolitical\", \"conservative\", \"centrist\", \"liberal\" buckets, plus two buckets that start Centrist and head Left or Right in the presence of overuse. One bucket per thingy. Hey, it's not an exact science. But it should help set the rough foundation for the more detailed use cases below.
Non-political Stuff: Algorithms, data structures, concrete mathematics, complexity analysis, information theory, type theory, computation theory, and so on. Basically all CS theory. These disciplines occasionally inspire tempest-in-a-teapot butthurtedness in academia, but when it happens, it's just similar fish in too small a tank biting on each other. It's to be expected. Overall, these essentially mathematical disciplines are timeless, and they are all equally applicable to both the Liberal and Conservative programming worlds. Yes, even type theory.
Conservative Stuff: Provably sound type systems. Mandatory static type annotations. Nonpublic symbol visibility modifiers (private/protected/friend/etc.). Strict, comprehensive schemas. all-warnings-are-errors. Generics and templates. Avoidance of DSLs (XPath, regexps) in favor of explicit DOM manipulation and hand-rolled state machines. Build dependency restrictions. Forced API deprecation and retirement. No type equivalence (i.e. no automatic conversions) for numeric types. Checked exceptions. Single-pass compilers. Software Transactional Memory. Type-based function overloading. Explicit configuration in preference to convention. Pure-functional data structures. Any kind of programming with the word \"Calculus\" in it.
Centrist (or flat-out Neutral) Stuff: Unit testing. Documentation. Lambdas. Threads. Actors. Callbacks. Exceptions. Continuations and CPS. Byte-compilation. Just-in-time compilation. Expression-only languages (no statements). Multimethods. Declarative data structures. Literal syntax for data structures. Type dispatch.
Liberal Stuff: Eval. Metaprogramming. Dynamic scoping. all-errors-are-warnings. Reflection and dynamic invocation. RTTI. The C preprocessor. Lisp macros. Domain-specific languages (for the most part). Optional parameters. Extensible syntax. Downcasting. Auto-casting. reinterpret_cast. Automatic stringification. Automatic type conversions across dissimilar types. Nil/null as an overloaded semantic value (empty list, empty string, value-not-present). Debuggers. Bit fields. Implicit conversion operators (e.g. Scala's implicits). Sixty-pass compilers. Whole-namespace imports. Thread-local variables. Value dispatch. Arity-based function overloading. Mixed-type collections. API compatibility modes. Advice and AOP. Convention in preference to explicit configuration.
Centrist Stuff that Becomes Conservative If Taken Far Enough: Type modeling. Relational modeling. Object modeling. Interface modeling. Functional (i.e., side-effect-free) programming.
Centrist Stuff that Becomes Liberal if Taken Far Enough: Dynamic class loading and dynamic code loading. Virtual method dispatch. Buffer-oriented programming.
Woah, that exercise was surprisingly fun! It's far from complete, but hopefully you get the idea.
Some natural themes arise here:
-- implicit is generally liberal; explicit is generally conservative.
-- performance-oriented is generally conservative; late-optimized is generally liberal.
-- compile-time binding is generally conservative; runtime/late binding is generally liberal.
-- concurrency and parallelism in general seem to be politically charged topics, but the disagreement is orthogonal to the liberal/conservative camps.
I'd love to keep going with the classification. But I'll stop here, since we've got higher-level stuff to discuss.
I'll walk you through a bunch of examples to show you how just widespread and deep-rooted this political phenomenon is.
Here are some very rough categorizations. Note that within each language camp there are typically liberal and conservative sub-camps. But as a whole, language usage tends to be dominated by what the language makes possible (and easy), so the culture tends to follow the features.
This list is just a few representative examples to give you the flavor. I'm only listing general-purpose languages here, since DSLs and query languages are typically feature-restricted enough to be hard to categorize.
Assembly language: Batshit liberal.
Perl, Ruby, PHP, shell-script: Extremist liberal.
JavaScript, Visual Basic, Lua: Hardcore liberal.
Python, Common Lisp, Smalltalk/Squeak: Liberal.
C, Objective-C, Scheme: Moderate-liberal.
C++, Java, C#, D, Go: Moderate-conservative.
Clojure, Erlang, Pascal: Conservative.
Scala, Ada, OCaml, Eiffel: Hardcore conservative.
Haskell, SML: Extremist conservative.
These are my own categorizations based on my own personal experiences with these languages and their respective communities. Your mileage may vary. However, I'd be quite surprised if you chose to move any of these languages more than a step or two away from where I've positioned it.
One thing that jumps out is that a language doesn't have to be statically-typed or even strongly-typed in order to be conservative overall. More on that in a bit.
The next thing you might notice from the list is that the liberal and moderate languages are all pretty popular, and that popularity declines sharply as languages head into conservative territory.
I think this has a simple explanation: It's possible to write in a liberal language with a conservative accent, but it's very hard (and worse, discouraged) to write in a conservative language with a liberal accent.
For instance, it's straightforward to write JavaScript code in a way that eschews reflection, eval, most automatic type casting, prototype inheritance, and other dynamic features. You can write JavaScript that plods along as unadventurously as, say, Pascal. It doesn't have all the static type annotations, but you can replace them with assertions and unit tests and stereotypically stolid code organization.
But if you try writing your Haskell code with a bunch of dynamic features, well, you're in for a LOT of work. Haskell enthusiasts have managed to implement dynamic code loading and a ton of other ostensibly dynamic features, but it was only through herculean effort.
What's more, if you write your liberal-language code in a conservative way, people will just look at it and say: \"Well, it's kinda boring, and you could have saved a lot of coding by using some dynamic features. But I guess it gets the job done. LGTM.\"
Whereas if you write your conservative-language code in a liberal way, you run the risk of being ostracized by your local language community, because... why are you doing all that dangerous dynamic stuff in the first place? I'll explore this cultural phenomenon further when I talk about Clojure below.
The last big, interesting observation from the list is that a lot of the most popular languages out there are only moderately conservative -- even if they think of themselves as quite conservative compared to their ultra-dynamic cousins.
I've said it before, and it bears repeating here: the reason C++, C# and Java have been particularly successful in the marketplace is that -- just like effective politicians -- they know how to play both sides.
C++ allows liberal-biased programmers to program in straight C, and it allows conservative-biased programmers to layer in arbitrary amounts of static type modeling, depending on how much work they want to expend in order to feel secure. Java? Pretty much the same story.
Playing to both the fast-and-loose and lock-your-doors mindsets has proven to be a key ingredient to market success. Also marketing, but it helps a LOT to be viewed as philosophically friendly by both the liberal and conservative camps.
There is a new crop of languages on the horizon (for instance, Google's Dart language, but also new specs for EcmaScript) that are deliberately courting the centrist crowd -- and also delicately playing to grab both the liberals and conservatives -- by offering optional static types. In principle this is a sound idea. In practice I think it will come down to whether the marketing is any good. Which it probably won't be.
Language designers always seem to underestimate the importance of marketing!
Just for fun, let's contrast four similar-ish tech companies in their software-political outlook.
Facebook -- Diagnosis: Extremist Liberal. Despite their scale, they are still acting like a startup, and so far they've been getting away with it. They use primarily C++ and PHP, and they're prone to bragging about how their code calls back and forth from PHP to C++ and back into PHP, presumably bottoming out somewhere. Their datastore is memcached: just name-value pairs. No schema. They dump the data and logs into a backend Hive store and run Hadoop mapreduces for offline data analysis. They still hold all-night hackathons every other week or so, which will remain feasible for them as long as the majority of their programmers are very young males (as was the case last time I toured there) and their stock continues to promise great riches (as was not so much the case last I checked.) As a company they are tightly knit and strongly biased for action, placing a high value on the ability of individual programmers to launch features to their website with little to no bureaucracy or overhead. This is pretty remarkable for a company as big as they are, with as many users as they have. Conservatives no doubt regard them with something between horror and contempt. But Facebook is proving that programmers of any world-view can get a hell of a lot accomplished when they gang up on a problem.
Amazon.com -- Diagnosis: Liberal. Which is surprising, given how long they've been in business, how much money is at stake, how mature their Operations division is, and how financially conservative they are. But \"Liberal\" is actually quite a retreat compared to their early days. Back in 1998-1999 they were almost exactly like Facebook is today, with the sole exception that they put everything in relational databases and did a ton of up-front relational data modeling. Well, except in Customer Service Apps, where we used a name/value store just to be flexible enough to keep up with the mad chaotic scramble of the business launches. All part of my multi-decade indoctrination as a Liberal. In any case, despite many corporate improvements with respect to work-life balance (which happened after several stock plunges and years of significant double-digit turnover in engineering), Amazon has retained its liberal, startup-like engineering core values. Every team owns their own data and makes their own decisions, more or less like independent business units. Amazon still launches and executes faster than just about anyone else out there, because they're still willing to take real risks (incurring occasional huge outages), and to make hard decisions in favor of launching early and often. Above all else, Amazon has proven conclusively that after fifteen years, they can still innovate like nobody else. They've still got it.
Google -- Diagnosis: Conservative. They began life as slightly liberal and have grown more conservative ever since. Google was only software-liberal in the very very early days, back when the search engine itself was written in Python. As they grew, they quickly acquired a software conservatism driven entirely by the engineers themselves. Manifestos were written about the dangers of using multiple languages, and strict style guides were put in place to severely limit \"risky\" or \"hard to read\" language features of the few languages they did allow. Google's JavaScript code is written in an extremely conservative style with extensive static type annotations, and eval is forbidden. The Python style guide forbids metaprogramming and other dynamic features, which makes their Python look a lot like untyped Java. And they have severely limited the use of many C++ language features, with C++11 support rolling out literally one feature every few weeks. (There are over five hundred new features in C++11.) In internal surveys, Google engineers commonly cite bureaucracy, churn and complexity as core obstacles to feature advancement and rapid launches. Google has made serious attempts on several occasions to reduce this bureacracy, but they always get pushback from -- surprise -- the engineers themselves, who have grown so staunchly conservative that they actively (and even more often, passively) resist the introduction of more flexible stacks and technologies. Most of the major technological shifts within Google over the past half-decade have been overtly conservative. For a liberal like me, it has been a very sad process to observe. But at least I've found myself a niche that's widely regarded (by both camps) as valuable, and within my own org we can still be pretty liberal and get away with it.
Microsoft -- Diagnosis: Batshit Conservative. Microsoft has two geese that lay golden eggs: Office and Windows. Microsoft has been reduced to a commercial farmer protecting the geese from all incursions. The golden eggs still have value, because customers are locked into the platform by the cost-ineffectiveness of retraining their fleets. But Microsoft can no longer innovate in Office or Windows precisely because of those corporate fleet retraining costs. Their OEMs are stretched as thin as they can go. Apple is dominating the handheld markets, and Microsoft is actively stifling their own innovation in Windows Phone because they're afraid it will cannibalize their core Windows business. Microsoft has not had a successful product-level innovation in fifteen, maybe twenty years. All of their successful products have been copies of competitors' products: IE, XBox, C#, .NET, Bing, Windows Phone, and so on ad infinitum. All great implementations of someone else's ideas. Microsoft's playbook is to embrace, extend, and leverage their brand to crush the competition -- or at least it was, until the goverment put an end to that circa 2002. Now the company genuinely doesn't know what the fuck to do with themselves, and what's more, instead of Bill Gates they now have a lunatic in charge. Employees are leaving in droves, all citing the same internal \"existential crisis\" and unbearable corporate politics caused by competing business units actively sabotaging one another. Microsoft has turned into a caricature of right-wing corporatism: sitting on their front porch with a shotgun cursing at passers-by, waiting for their government bribes to give them another few years of subsidies and shelters while they wait to die. I've personally chatted with close to four hundred current and ex-Microsoft employees over the past seven years. Oh, the stories I could tell you... someday, maybe.
Bonus company: Apple. Diagnosis: no idea, but they're so good at marketing that it's almost irrelevant. Would love to have more insight into their internal software culture, though. Any takers? Throwaway accounts? AMA?
OK, that was a fun exercise too. But we need to move on! Almost done now.
I've been meaning to follow up on Clojure for a while now. Over a year, at least. But until recently I didn't have the conceptual tools to explain what I wanted to say about it.
Now I do!
Clojure is a new-ish Lisp dialect that runs on the JVM and .NET, and I was honored to write the Foreward to \"The Joy of Clojure\" a while back. For a few years I had been really excited to start learning Clojure, and my initial experiences with it were quite positive.
However, I eventually learned that the Clojure community is extremely conservative. That is is pretty unusual for a Lisp dialect. Lisp is widely regarded as one of the most liberal language families in existence. And Clojure has the superficial appearance of being a laissez-faire kind of language. It is quite expressive, including a -- ahem -- liberal dose of new syntax. And it eschews static type annotations and strong type modeling in favor of a small set of highly regular, composable data types and operations -- not unlike, say, Scheme or Python.
But the resemblance to a liberal language ends there. Clojure's community came pre-populated with highly conservative programmers from the pure-functional world: basically Haskell/ML types (lots of puns today!) who happen to recognize the benefits of Lisp's tree syntax. So under its expressive covers, everything about Clojure is strongly conservative, with a core overriding bias towards protecting programmers from mistakes.
And the community follows suit. At a Clojure conference last year (or was it two years ago? time flies so fast these days...), there was a key presenter doing a talk on how Macros were basically harmful and should be avoided in modern Clojure code.
I trust that if you know anything about Lisp, your blood is basically boiling at this point. I know mine was.
But his argument is perfectly valid from the classic software-conservative's viewpoint. Macros allow you to invent domain-specific language abstractions. Those require documentation in order for users to figure out what they do, and what they mean. That means you can know Clojure and not really know enough to read someone's Clojure code without some documentation handy. Which is sort of the definition of a newbie. And there you have it. In a very real sense, conservatives fear being turned -- in the blink of an eye -- from masters into newbies by the application of macros.
It's sort of scary if you have a lot of your personal identity invested in knowing some shit. And wouldn't you know it, real-world politics conservatives are shown in study after study to be \"staunch\" in sticking to their world views rather than compromising. That means they have a lot of identity tied up in those views.
So while I liked a lot of what I saw in Clojure, as a hardcore software liberal I was inevitably and inexorably driven away from the language. And that's good for me, and it's good for Clojure. I mean, why should they compromise?
I think that my conceptual framework gives us an \"out\" -- a way to avoid being emotional about these subjects. Casting the problem as a clash between Liberalism and Conservsatism gives us the ultimate ticket for \"agreeing to disagree\".
Hopefully it will also help language designers and communities do a better job of targeted marketing. Right now just about every language out there makes a claim along the lines of \"This language is the best choice for everybody!\" But now we know that is very unlikely to be true -- or if it is, we can at least be assured that they're centrists, and they run the risk of being equally distasteful to everybody.
In the conservative/liberal framework, language designers can make more accurate, less bait-and-switchy claims; for instance: \"Haskell is the best choice for every radical extremist conservative programmer!\"
Well, we can work on the wording. But you get the idea.
I was thinking of going through a bunch more examples and stuff, but I see that I'm on my third (Editor's Note: fourth) glass of wine, which means my typing is about to give out any minute.
So let's wrap it up!
There's one kinda key point I wanted to get across, but didn't see a good place for it. That point is this: please do not be alarmed that I am calling you a (software) Conservative.
I worry that politically left-leaning programmers will hear the term \"conservative\" and will immediately associate it with all of the... uh, politically-charged connotations associated with far right-wing conservatism in the United States political arena today. You know, racism, sexism, religious fundamentalism, homophobia, warmongering, bear-shooting, that kind of thing.
I'm not saying they're bad, at least not in this essay. I'm just saying nobody in their right mind wants to be associated even remotely with those embarrassing wingnuts. See what fine-grained, nuanced distinctions three (Editor's Note: four) glasses of wine can produce? But I'm not saying their views are bad. No. Not here. I'm just observing that they're heavily politically charged viewpoints which have, for better or worse, recently come to be associated with the term \"conservatism\" in US politics.
So please do me a favor and try to dissociate those specific agendas and real-world political viewpoints from the generic term \"Conservative\", which here really just means \"risk averse\".
It's perfectly OK, and normal, to be a programming conservative. You don't have to shoot any bears. I would actually like to see the terms \"liberal\" and \"conservative\" to become badges of honor in the programming world. People should stand behind their beliefs. I mean, we do already, I think, so it shouldn't be much of a stretch to allow our beliefs to be given convenient labels.
Ah, me. I can see the Euphemism Treadmill rearing its ugly head already. We'll see.
Anyway, tell me what you think! I welcome any and all viewpoints and comments. Even from bears!
Special thanks to Writer's Block Syrah for reviewing this post for career suicide.
(Special Note to my fellow Googlers: Yes, I meant to post this externally. BOTH times. No, I am not the Mouth of Sauron.)
", "url": "https://whoislewys.com/blog/2019/09/06/Notes-From-The-Mystery-Machine_Bus", "title": "Notes from the Mystery Machine Bus", "summary": "A mirror of Steve Yegge's old Google+ post", "date_modified": "2019-09-06T12:00:00.000Z", "author": { "name": "Steve Yegge" }, "tags": [ "code", "rant" ] }, { "id": "https://whoislewys.com/blog/2018/10/18/audio_norm_guides_are_evil_and_immoral", "content_html": "There are SEVERAL guides online telling you how to normalize audio to provide a great listening experience by using FFmpeg's \"loudnorm\" filter. Most walk you through a simple process called \"track normalization.\" You set a goal loudness, and adjust every piece of audio up or down to match that loudness. This is evil and immoral and I'll tell you why:
Loudnorm changes the way things sound!
Loudnorm crushes dynamics!
The right way™ is \"Album Normalization,\" where every piece of audio (song, level soundtrack, etc.) gets normalized in relation to the loudest piece of audio.
\"With album normalization, just the loudest tracks of an album are made equally loud and the other tracks keep the relative level they had on their album.\" - Eelco Grimm, loudness researcher
If you're a developer creating a VR experience, or a game, or something where music features heavily, you need to do normalization, and you need to do it right.
You can't just normalize every piece of audio to a certain average level and call it good.
Not normalizing audio = poor user experience
There can still be jarring sound transitions. You ever watch TV at a comfy volume, realize the show is going to commercials, and feel dread flood through your body, ears recoiling in fear because you know the commercials you're about to see will be 10x as loud as the show?
Yeah, good normalization fixes that.
So unless you want your users to hate you, normalize your audio properly!
Now what happens if you normalize your audio wrong, crushing the dynamic range?
Honestly, a lot of people probably won't even notice (circa 2018).
But once good normalization becomes the standard for all experiences including audio, people WILL hear the difference between a great normalization algorithm and the loudnorm kludge copy-pasted from Stack Overflow, and they will HATE it. And don't you want your users to have the BEST experience?
\"Alright tough guy,\" you say, \"what makes you so qualified to talk about audio like this?\"
Well, I'm really not. I have very little ethos here. But I'm STILL positive users love properly normalized audio.
Why?\nCause TIDAL hired a smart guy, with a ton of audio street cred, to do a study on best normalization practices. He sent non-normalized & normalized versions of a playlist to 38 people. And guess what?\n80% of people preferred the album normalized versions to track normalized! Link to study. And those 80%, 9% said they'd never accept track normalization (an inferior normalizing method) if it were turned on by default!
Ok, I admit at least one of subjects was an audio nerd, as Ian Shepherd (legendary mastering engineer) admits he was a part of the study in his great post on the subject. Regardless, 31/38 people is a pretty strong majority. I imagine the study would translate well to a larger sample of non audio nerds as well.
Be careful following guides online dealing with subjects you're not familiar with. It can be tempting to just do what Google's Loudness page says, without realizing they're giving recommendations for Google Assistant developers. It can be easy to copy-paste a command from Stack Overflow, when the use case that was asked about is totally different from your own. You're used to these kinds of resources, and so am I.
But when it comes to writing lines of code that affect your entire project in a domain you don't know much about, you need to reach out to someone with knowledge in the field.
", "url": "https://whoislewys.com/blog/2018/10/18/audio_norm_guides_are_evil_and_immoral", "title": "Why FFmpeg Loudness & Normalization Guides Are All Wrong, Evil & Immoral!", "summary": "The Wrong Way", "date_modified": "2018-10-18T06:07:00.000Z", "author": { "name": "Luis Gomez", "url": "https://github.com/whoislewys" }, "tags": [ "DIY", "music", "audio" ] }, { "id": "https://whoislewys.com/blog/2018/10/10/the-only-4-deep-learning-frameworks-that-matter", "content_html": "Let me get this off my chest right off the bat: Keras is all you need for 90% of neural network applications. Before jumping in, you should understand what is going on with neural networks(watch 3Blue1Brown's videos).
Ideally, you can understand all the math behind them and code a basic one from scratch (read Nielsen's Neural Networks and Deep Learning).
But for most cases, even understanding is unnecessary with all the pretrained models frameworks ship with nowadays.
But, assuming you're not a n00b and you need to do more beyond just classifying 2D images or recognizing characters, here are the Deep Learning/ Neural Network frameworks can best fit your use case.
TLDR: Are you a researcher/startup employee that wants to test ideas 10x faster than the competition? Just use Keras already.
Okay okay fellow pedants - Keras is more of a library than a framework. Regardless of what you call it, it is undeniably an incredible tool. It's favored by researchers and AI startups alike for its readability and ability to rapidly create neural networks.
It works by using a framework in its backend (e.g. TensorFlow) to do all the neural network math, and wraps different functionalities for easy usage. Keras gives you high level functions to do things like create layers, apply max pooling, check accuracy, and other common neural network tasks in a single line of code.
Having these possibilities at the tips of your fingers is invaluable when trying out many different model architectures.
pro deployment tip, you can export Keras models as TF graphs and upload them to CloudML to deploy your network in under 30 mins.
TLDR: Do you need a neural network to run on mobile? Move Caffe2 to the top of your list.
This Facebook-supported framework offers extremely high performance with a focus on large scale deployments and mobile models. Users love it for its excellent performance on mobile devices, flexibility, and ease of writing. It is usually used through the Python API, though there is a C++ API as well. Also, a model can be written in Python then deployed in pure C++ for additional performance and scalability.
Caffe2 also comes with a TON of pretrained models form the lovely Model Zoo.
TLDR: Need in-browser neural networks?
Sorry to put you at the bottom of the list, TensorFlow. The focus on parallelism and ease of integration with Google's CloudML makes TensorFlow deploying highly scalable networks easy, at the expense of typically writing more complex code. It is not as intuitive to write as the other frameworks mentioned. The main reason I list it here because of the exciting new TensorFlow.js enabling high-performance, in-browser models. It is getting better and better at mobile performance but still doesn't touch Caffe2. I think TensorFlow is worth keeping an eye on as Google continue to innovate and integrate ML into different areas, but unless you have
if you're a beginner who needs to deploy a model, CloudML is a great option. But don't feel like you have to learn TensorFlow. You can write and train a model in Keras and export a TensorFlow graph and upload that.
There you go. In my humble opinion, these are the only 4 deep learning frameworks that matter in 2018.
Don't let yourself get into a black hole of paralysis by analysis, researching every framework endlessly. Pick one of the above. There are good tutorials for all of them. Pick one and learn it. If you need to switch over to another for some reason, the skills you learn in one will transfer to other. Just start!
", "url": "https://whoislewys.com/blog/2018/10/10/the-only-4-deep-learning-frameworks-that-matter", "title": "The Only 4 Deep Learning Frameworks That Matter", "summary": "Let me get this off my chest right off the bat: Keras is all you need for 90% of neural network applications. Before jumping in, you should understand what is going on with neural networks(watch 3Blue1Brown's videos).", "date_modified": "2018-10-10T15:27:00.000Z", "author": { "name": "Luis Gomez", "url": "https://github.com/whoislewys" }, "tags": [ "AI", "ML", "Code", "Deep Learning" ] }, { "id": "https://whoislewys.com/blog/2018/06/25/Building My Own Guitar", "content_html": "*there is no red K emoji, 🅱️ is the best I can do
I wanted a faster, more efficient way to make music.
My solution? Stick a Launchpad in a guitar
Sadly, no electric guitars I found had the room for a launchpad.
My solution (pt 2)? Build my own guitar with correct dimensions.
My advice? DON'T DO IT. It was really hard and time consuming.
Unless you NEED to do this to get specific dimensions & realize some crazy idea of your own,\njust buy a nice used guitar. You'll still get a lot of instrument for your money, and SAVE TIME.
Do not delay, sing and play everyday.
Total Cost: $170
Thankfully my pops had most of these tools laying around. Did have get a cheap jigsaw from harbor freight.
All these tools are very easy to use, and very easy to misuse. Be careful.
Don't get so caught up in planning on buying specific tools or brands unless you really know what you want.
Otherwise you'll spend hours and hours on woodworking forums, guitar forums, electronics forums, and YouTube thinking and spectating instead of doing.
Measure twice, cut once.
When you're cutting out the body, don't try to trace the line exactly with the jigsaw. Drill holes near curves, and aim at the holes with the jigsaw. (You could even do this with a handsaw, save yourself some money!) Make the shape nicer using a file & rough sandpaper.
If you don't have a drill watch the video below titled 'Relief Cuts'.
If you don't have a soldering iron and solder laying around, buy a pre-wired harness from guitarfetish(GFS). Also, you can make good joints using electrical tape and zip ties, see the video linked below 'Joining Wires with NO Solder'
Joining Wires with NO Solder
\nhttps://www.youtube.com/watch?v=f0Gierk4SCw
\nTLDR: Use Electrical tape to join the wires, zip tie around the electrical tape.
Relief Cuts
\nhttps://www.youtube.com/watch?v=5qCY-iXPvw8
What Can You Do With A Jigsaw?
\nhttps://www.youtube.com/watch?v=jWneDzqAbH8
\nSmoother cuts on the side of the board the teeth are immediately facing (upward pointing blades make smoother lines on underside of board)
Cutting A Guitar Body Out With Hand Tools (2X speed this guy speaks slow)
\nhttps://www.youtube.com/watch?v=2Lm3l_GOAOQ
Reducing Hum
\nhttps://www.youtube.com/watch?v=FI_v-NjBRKc
[My custom plan](/downloads/luis paul jr schemes 4 WEB.zip)
I adapted my plan from here: Electric Herald High Quality Guitar Templates
", "url": "https://whoislewys.com/blog/2018/06/25/Building My Own Guitar", "title": "How I Built My Own Guitar", "summary": "With hand tools and NO woodworking experience", "date_modified": "2018-05-23T12:00:00.000Z", "author": { "name": "Luis Gomez", "url": "https://github.com/whoislewys" }, "tags": [ "DIY", "Music", "Audio" ] }, { "id": "https://whoislewys.com/blog/learn-audio-compression", "content_html": "Here are quick links to each section. If you don't know where to start, go to Downloads.
Downloads
\nVideo
\nTranscription
\nCompressor Controls
\nThreshold
\nRatio
\nAttack
\nRelease
\nExample Settings
\nFIN (comment section)
Download the FREE Compressor Cheatsheet for an easy, one-page summary of compressor fundamentals.
Note: the PDF above is A4 sized & optimized for web.
\nFor a great looking printout and files good for print, please go to the shop.
Print & plaster it on your wall, gift it to your producer, make it your phone background, whatever.
If you want to go deeper and really get a feel for each compressor control, then download this drum loop and free compressor to follow along with the video below!
Drum Loop (Right click ‚ --> Save Link As)
Video embed will go here
Welcome to this video, you are very brave. I'm going to help you get a feel for how a compressor works without making it complicated. You're intuitively going to know what the controls do.
I'm also gonna link you to one of the best sounding free compressors and give you a couple example settings for adding punch and thickening your tracks.
Let's do it.
First we'll cover the fundamentals of compression:
It changes the shape of audio signals by reducing volume. This 'reducing volume' is also called gain reduction. Same thing. The two most common uses of a compressor are to make audio punchier, and to make audio 'thicker' or more even.
So say we want to make our drums punchier. How do we do this by reducing volume? We leave the volume alone for the initial drum hit, then we reduce the volume right after. Then bring the volume up just in time for the next drum hit. This gives you a punchier sound.
We can also make the drums sound more even, or thicker or fatter if you like. For this, we turn down the volume for the initial punch/ poke of each drum hit and turn the volume back up to normal quickly. This will smooth out the initial hit of the drums, leaving you with a more even signal with more body. The drum tone stays present or sustains for a longer time
So, how do we make a compressor do this for us?
To understand compression, we just need 4 simple controls. I recommend you follow along, there are FREE download links for the drum loop and compressor in the description and also a handy and beautiful Compression Cheatsheet with all this info on it, but if you just wanna watch that's cool too we're gonna learn regardless.
That's all the controls. Not that scary right?\nLet's apply our knowledge and wrap this little lecture up with some example settings for punch and settings for thickness / even-ness
Alright so the theory is great. But what numbers should I look for?
If I want to make my vocals sound thicker and more 'in your face,'
If I want to make bone crushing drum sounds how do I do that?
Real quick, it all depends on the track. So DO NOT just copy my settings here. Use them as a starting point. The only way to master compression is learn what the controls do and play around.
Also, make sure you like the balance of all your tracks. If the volumes of your tracks out of a wack, compression will not help! Get a balance you like just by adjusting the faders.
Alright that said, let's get to the settings.
Lots of people talk about using compressors to add punch. Here are some typical settings and what it sounds like.
We're gonna look for 10db of gain reduction on our meter right here.
We're gonna use an 8:1 ratio to really clamp down on the drum sound after the initial hit. We've got an attack time of 20 to reduce the tail right after each hit, and release of 150 so the volume recovers a bit in between hits.
Finally, threshold depends on the volume of your input audio so I'll just set to wherever I start seeing around 10db of gain reduction.
This probably isn't something you'd want to send out, but I'm making the effect very dramatic to demonstrate what 'punch' sounds like from a compressor.
Hear how the medium attack time lets the very beginning of each drum hit through? If I reduce this the drums get clickier and clickier. If I slow down the attack, we start getting this weird sound. But right about 15-40 we get these nice beefy drum hits with little tail. You might wanna set this smaller for a faster tempo song, or larger for a slower tempo song.
We're gonna look for around 5db gain reduction.
Ratio we're going 4:1, attack 0.3ms, release 20ms.
Again, threshold depends on your input volume.
Hear how these really fast attack and release times will squash the initial hits of the drums and leave them with a lot of sustain
This makes the drums less pokey, more controlled and fatter. This also allows you to bring the volume up louder than you could have before compression.
One last note, It's human nature for us to think louder is better.
So when you're trying out different compressor settings be careful to not just toss a compressor in just because it makes your audio louder. A lot of presets do that.
Check meters in your DAW, listen carefully, and toggle the compressor on and off to make sure the volume is THE SAME before and after compressing.
Congratulations! That's all the fundamentals of compression, with some solid settings to get you started. Have fun with it
There is one common setting we didn't discuss cause we need to understand the others first: knee.
It can make compression sound more natural. If you want to know how it works, There's a link in the description to an Ableton Forum thread that describes it beautifully and simply - it's also described briefly in the Compression Cheatsheet.
Leave a like if you liked, if you have a question or a request or you think I suck leave a comment, if you'd like more subscribe to the channel. That's it Peace! 😊