Skip to main content

NodeJS FAQ

  • 持久化 DB 存储是 NodeJS 的弱项
    • 官方 正在尝试支持集成 SQLite
    • 纯 JS 没有好的存储实现
    • SQLite 依赖 addon
    • SQLite over WASM 还不成熟
  • FFI 是 NodeJS 的弱项
    • 可以尝试 bun:ffi
    • nodejs/node#46905 ffi implementation
    • nodejs/node#46233 Reconsider adding FFI to the core
      • tinycc does not support all of the platforms that node does
  • 直接运行 typescript
    • --experimental-strip-types
  • import.meta.main

env​

  • NODE_OPTIONS

Tool Chain​

  • 静态项目 - ViteJS
  • 前端项目 - NextJS+trpc+NestJS - 复杂项目
    • 部分 NestJS 逻辑共用
    • 输出 standalone 模式 - 直接拷贝到容器即可
  • 后端项目 - NestJS+fastify
    • 开发: ts-node --swc --esm --transpileOnly --watch src/main.ts
      • tsx 不支持 decorator，不然可以用 tsx
    • 构建: esbuild --bundle --external=sharp src/main.ts
      • 输出一个 js 放到容器即可
      • 注意加 tsc 插件处理 decorator
  • 基础依赖
    • zod
    • valtio, zustand
    • daisyui, styled-components
    • dayjs
pnpm node --loader ts-node/esm --watch ./src/apps/ve-contract-server/main.ts
tsconfig.json
{
  "ts-node": {
    "transpileOnly": true,
    "swc": true,
    "esm": true,
    "files": true,
    "experimentalSpecifierResolution": "node"
  }
}

Node Dev​

  • ts+watch - tsx, ts-node, swc-node
    • +decorator
      • ts-node+swc
    • +tsconfig.paths
      • tsx, ts-node+tsconfig-paths
      • +esm
        • tsx
  • 目前还没有 esm+tsconfig.paths+decorator 的

选择包管理器​

选择 pnpm

  • pnpm
    • 速度快
    • 使用 hardlink - 节省空间，速度快
      • 如果在 wsl 可能用不了 hardlink，回退为 softlink 或者 copy 方式
    • 意外情况比较少
    • workspace 支持完善
  • npm
    • node_modules 太大
    • 安装慢
    • 如果 pnpm 不兼容的情况可以考虑 npm
  • yarn v1
    • 比 npm 好一点 - 区别越来越小
    • 支持 workspace
  • yarn berry
    • 很多工具不支持
    • pnp 方式生态推进慢
    • 不建议使用

shebang​

CJS

#!/usr/bin/env node

ESM

#!/usr/bin/env bash
":" //# comment
exec /usr/bin/env node --input-type=module - "$@" < "$0"

NodeJS 20.10+

node --experimental-default-type=module cli
node --input-type=module --eval 'console.log("Hello, world!")'

performance​

  • UV_THREADPOOL_SIZE=4
    • 最大 1024
  • knex
    • pool min:2, max: 10
  • fastify
  • fast-json-stringify
  • find-my-way

musl​

sqlite3​

# ./node_modules/sqlite3/build/Release/node_sqlite3.node
find ./node_modules/sqlite3/ -iname '*.node'

npm install sqlite3 --build-from-source

source map​

import '@cspotcode/source-map-support/register';

cjs __dirname​

import * as url from 'url';
const __filename = url.fileURLToPath(import.meta.url);
const __dirname = url.fileURLToPath(new URL('.', import.meta.url));

直接执行 typescript 或 esnext​

  • tsx
  • ts-node
  • babel
  • tsm
# tsx
# ==========
npx tsx app.ts
node --loader tsx app.ts
node --loader @esbuild-kit/esm-loader app.ts

# ts-node
# ==========
node -r @ts-node/register app.ts
# tsconfig 里的 path 能生效
node -r @ts-node/register -r tsconfig-paths/register app.ts

# babel
# ==========
node -r @babel/register app.js

Package 'OpenEXR', required by 'vips', not found​

找不到 brew 安装的 pkgconfig

PKG_CONFIG_PATH=/usr/local/opt/vips/lib/pkgconfig:/usr/local/opt/glib/lib/pkgconfig:/usr/local/opt/openexr/lib/pkgconfig:/usr/local/opt/imath/lib/pkgconfig npm i
/usr/local/include/vips/vips8:35:10: fatal error: 'glib-object.h' file not found
# export CC
export CXX="$(which g++) -I/usr/local/opt/glib/include/glib-2.0/ -I/usr/local/opt/glib/lib/glib-2.0/include/"

export CXX="$(which g++) $(pkg-config --cflags glib-2.0)"

libtool: unrecognized option -static when building​

libtool:   error: unrecognised option: '-static'
brew unlink libtool
rm -rf /usr/local/bin/libtool
which libtool

# 如果添加了 /usr/local/opt/libtool/libexec/gnubin
export PATH=$(echo $PATH | sed -r 's/:[^:]*?libtool[^:]*:/:/')

require() of ES modules is not supported​

尝试降级依赖

NodeJS v18 fetch proxy​

  • 不支持 Agent
  • 用 node-fetch 或者 undici
const Undici = requuire('undici');
const ProxyAgent = Undici.ProxyAgent;
const setGlobalDispatcher = Undici.setGlobalDispatcher;

setGlobalDispatcher(new ProxyAgent(process.env.HTTP_PROXY));

RequestInit: duplex option is required when sending a body​

TypeError: Cannot set property duplex of #<Request> which has only a getter
let init: RequestInit & Record<string, any> = {
  get duplex() {
    return 'half';
  },
};

Critical dependency: require function is used in a way in which dependencies cannot be statically extracted​

检查下是不是 import 路径错误，可能因为 IDE 自动导入，指向了错误路径。

Undefined variable module_name in binding.gyp while trying to load binding.gyp​

可能 npm 问题，使用 pnpm 构建没问题

Custom ESM Loaders is an experimental feature. This feature could change at any time​

目前无法 supress 警告, 只能通过 require 注入避免

node --require suppress-experimental.cjs --loader tsx app.ts
suppress-experimental.cjs
'use strict';
// When using the ESM loader Node.js prints either of the following warnings
//
// - ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
// - ExperimentalWarning: Custom ESM Loaders is an experimental feature. This feature could change at any time
//
// Having this warning show up once is "fine" but it's also printed
// for each Worker that is created so it ends up spamming stderr.
// Since that doesn't provide any value we suppress the warning.
const originalEmit = process.emit;
// @ts-expect-error - TS complains about the return type of originalEmit.apply
process.emit = function (name, data, ...args) {
  if (
    name === `warning` &&
    typeof data === `object` &&
    data.name === `ExperimentalWarning` &&
    (data.message.includes(`--experimental-loader`) ||
      data.message.includes(`Custom ESM Loaders is an experimental feature`))
  )
    return false;

  // return originalEmit.apply(process, arguments as unknown as Parameters<typeof process.emit>);
  return originalEmit.apply(process, arguments);
};

Reached heap limit Allocation failed - JavaScript heap out of memory​

NODE_OPTIONS="--max-old-space-size=8192" eslint --fix
  • DEBUG=*
  • DEBUG=typescript-eslint
  • TIMING=1

Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory​

  • 可增加
node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'
NODE_OPTIONS="--max-old-space-size=8192" node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'

Can't resolve 'pg-native'​

// webpack
export default {
  resolve: {
    alias: {
      'pg-native': path.join(__dirname, 'pg-native'),
    },
  },
};

pg-native

{
  "name": "pg-native",
  "private": true,
  "main": "index.js"
}
{
  "name": "your-module",
  "private": true,
  "dependencies": {
    "pg": "^8.3.3",
    "pg-native": "file:./modules/pg-native"
  }
}
module.exports = null;

REPL load​

node -i -e "$(< rc.js)"

error:0308010C:digital envelope routines::unsupported​

export NODE_OPTIONS=--openssl-legacy-provider

snapshot / compile​

  • 使用 snapshot 加速启动
    • 例如: main.mjs 10mb, main.mjs.map 20mb
      • --enable-source-maps 启动 40s
      • 无 enable-source-maps 启动 2s
  • NODE_COMPILE_CACHE 
echo "globalThis.foo = 'I am from the snapshot'" > snapshot.js
node --snapshot-blob snapshot.blob --build-snapshot snapshot.js
echo "console.log(globalThis.foo)" > index.js
node --snapshot-blob snapshot.blob index.js

echo "require('v8').startupSnapshot.setDeserializeMainFunction(() => console.log('I am from the snapshot'))" > snapshot.js
node --snapshot-blob snapshot.blob --build-snapshot snapshot.js
node --snapshot-blob snapshot.blob
bytenode --compile server.js
bytenode server.jsc

fastify vs express​

  • fastify
    • 更快
      • fast-json-stringify
    • 模块化
    • 更易用
  • express
    • 更成熟？
    • monolithic
    • 生态更大 - 功能更多

import npm global​

export NODE_PATH=$(npm root --quiet -g)

ERR_OSSL_EVP_UNSUPPORTED​

Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:71:19)
    at Object.createHash (node:crypto:133:10)
    at BulkUpdateDecorator.hashFactory (~/node_modules/.pnpm/webpack@5.39.0_webpack-cli@4.7.2/node_modules/webpack/lib/util/createHash.js:145:18)
  • pnpm 安装有问题，换 npm 后好了

getaddrinfo ENOTFOUND undefined​

fetch(`http://example.com`);

命令行获取 package 所在目录​

pnpm pwd
pnpm node -e 'process.stdout.write(path.resolve(__dirname))'
PKG_ROOT ?= $(shell pnpm node -e 'process.stdout.write(path.resolve(__dirname))')

ioredis vs redis​

⚠️ ioredis 在被 redis 收编后不在怎么维护

The operation failed for an operation-specific reason: Cipher job failed​

Failed to execute 'digest' on 'SubtleCrypto': 2nd argument is not instance of ArrayBuffer, Buffer, TypedArray, or DataView.​

多半是传入了 undefined

detect runtime​

  • bun 也会设置 process.versions.node
console.log(process.release.name); // 总是 node

// bun
if (process.versions.bun) {
}
if (typeof Bun !== 'undefined') {
}

unhandled​

  • uncaughtException
process
  .on('unhandledRejection', (reason, p) => {
    console.error('Unhandled Rejection at Promise', reason, p);
  })
  .on('uncaughtException', (err, origin) => {
    fs.writeSync(process.stderr.fd, `Caught exception: ${err}\n` + `Exception origin: ${origin}\n`);
    console.log(`ERROR`, err);
    process.exit(1);
  });

The "windows-1252" encoding is not supported​

Throug

new TextDecoder('latin1');
# icu-data-en 不够
# 11.53 MB / 29.38 MB
apk add icu-data-full

Download​

shell scripting​

#!/usr/bin/env -S npx tsx
import { Command } from '@commander-js/extra-typings';

const program = new Command()
  .name('word-count')
  .description('CLI to count words')
  .version('1.0.0')
  .argument('<file>')
  .parse();

RSA_PKCS1_PADDING is no longer supported for private decryption at privateDecrypt (native) at decrypt​

// 触发错误
crypto.privateDecrypt(
  {
    key: privateKey,
    padding: crypto.constants.RSA_PKCS1_PADDING,
  },
  buffer,
);

可以使用 node-rsa

  • 大约慢 10 倍
import NodeRSA from 'node-rsa';

const key = new NodeRSA(privateKey);
key.setOptions({ encryptionScheme: 'pkcs1' });
const decrypted = key.decrypt(buffer);

Error ShellJSInternalError: spawn EBADF​

{
  "errno": -9,
  "code": "EBADF",
  "syscall": "spawn"
}