Use npm link to install ep_etherpad-lite. This places a package.json

file in the root directory that references ./src directory as the file
source for `ep_etherpad-lite`.

Remove --legacy-peer-deps and --no-save when invoking npm. There is no
need for them anymore, as we are bumping npm now to v8.

./src/package.json contains all dependencies of Etherpad core
(package name ep_etherpad-lite) as before. The root directory's
package.json file references ep_etherpad-lite and also contains
references to any installed plugins.

Remove npm from package.json as we depend on a recent version now; PATH is still updated as before, so in the future we may install a custom npm version again

lint package-lock: update exception for sqlite3

remove node_modules and package.json during installDeps.sh

update Dockerfile

adapt minify

windows build

Fixed installOnWindows.bat

remove node_modules from git

bump minimal node/npm version in src/bin/functions.sh

add changelog notes

update installdeps

fix dockerfile

docker: test npm prefix set to the etherpad directory

workflow: upgrade-from-latest-release needs to be adapted until next release is out

Revert "docker: test npm prefix set to the etherpad directory"

This reverts commit b856a2488c9dbfb2acf35309cd1ee83016b631ad.

use npm link --bin-links=false to prevent it from copying bin files

temp fix for scripts as they are not installed to bin directory anymore

adjust bin paths in Dockerfile

Dockerfile

add hint for npm link, dockerfile

update dockerfile

Revert "Fixed installOnWindows.bat"

This reverts commit 70d0716bbedc4c0c1043155fcc5d157f01775c61.

try installOnWindows; still TODO: no difference between production and development; no warning like in installDeps.sh before update - it just removes package* and node_modules so admins must be aware of the plugins they want to reinstall later

update installOnWindows.bat

update package-lock.json

Dockerfile

Dockerfile

add file: scheme for lint check - needed as long as we have the plugin compatibility symlinks in ./src/node_modules

fix installOnWindows

upgrade-from-latest-release workflow: adapt cypress installation

src/package.json: test-container fix path to _mocha; maybe revert this in case we enable bin-links again

src/package.json: add test-on-windows script

another try with test-on-windows, without using bin-links

use bin-links on windows

Revert "use bin-links on windows"

This reverts commit f50ec2a9fabe3098d48e8f412b73c01edbe2140e.

invoke mocha binary on windows

run npm i once on windows, to make bin files available - why?

remove supertest on windows production builds

add symlink for mocha

debug

Revert "debug"

This reverts commit 8916a0515ca2897c57ca65fef49fd0b3610d2989.

Revert "add symlink for mocha"

This reverts commit 3c60bef77d2a120d24fce14421fe638598cd849d.

windows workflow: adapt cypress path

frontend admin tests

src/bin/buildForWindows.sh

@@ -48,6 +48,9 @@ try ./src/bin/installDeps.sh
 log "copy the windows settings template..."
 try cp settings.json.template settings.json
 
+log "Because this is a production build, we delete supertest"
+try rm src/node_modules/supertest
+
 log "resolve symbolic links..."
 try cp -rL node_modules node_modules_resolved
 try rm -rf node_modules

src/bin/functions.sh

@@ -1,10 +1,10 @@
 # minimum required node version
-REQUIRED_NODE_MAJOR=12
-REQUIRED_NODE_MINOR=13
+REQUIRED_NODE_MAJOR=16
+REQUIRED_NODE_MINOR=0
 
 # minimum required npm version
-REQUIRED_NPM_MAJOR=5
-REQUIRED_NPM_MINOR=5
+REQUIRED_NPM_MAJOR=8
+REQUIRED_NPM_MINOR=19
 
 pecho() { printf %s\\n "$*"; }
 log() { pecho "$@"; }

src/bin/installDeps.sh

@@ -37,23 +37,64 @@ if [ ! -f "$settings" ]; then
   cp settings.json.template "$settings" || exit 1
 fi
 
+log "Removing src/node_modules."
+rm -rf ./src/node_modules || true
+
+# try to determine if plugins were installed using --no-save
+ROOT_PLUGINS_EXIST=1
+for file in node_modules/*
+do
+  if [ ! -e "$file" ]; then break; fi
+  if [ -L "$file" ] && [ "$file" = "node_modules/ep_etherpad-lite" ]; then break; fi
+
+  if expr "$file" : "node_modules/ep_*" > /dev/null; then
+    ROOT_PLUGINS_EXIST=0
+  fi
+done
+
+PACKAGE_EXISTS=1
+PACKAGELOCK_EXISTS=1
+if test -f ./package.json; then PACKAGE_EXISTS=0;fi
+if test -f ./package-lock.json; then PACKAGELOCK_EXISTS=0;fi
+
+if [ "$PACKAGE_EXISTS" = "1" ] || [ "$PACKAGELOCK_EXISTS" = "1" ]; then
+  if [ "$ROOT_PLUGINS_EXIST" = "0" ]; then
+    log "You have plugins in ./node_modules but don't have a package.json or package-lock.json file."
+    log "Please manually remove your ./node_modules directory, run this script again and install any plugins with npm i ep_plugin1 ep_plugin2 afterwards"
+    exit 1
+  fi
+fi
+
+log "Linking src as new package ep_etherpad-lite."
+exit_code=0
+(cd ./src && npm link --bin-links=false) || exit_code=$?
+
+if [ "$exit_code" != 0 ]; then
+  log "npm link failed. If there was a permission error, please set a prefix for npm."
+  log "The prefix can be set e.g. with npm config set prefix $HOME/.npm-packages"
+  log "This will create a symlink in $HOME/.npm-packages/lib/node_modules that points to this directory."
+  exit 1
+fi
 
 log "Installing dependencies..."
-(mkdir -p node_modules &&
-cd node_modules &&
-{ [ -d ep_etherpad-lite ] || ln -sf ../src ep_etherpad-lite; } &&
-cd ep_etherpad-lite)
-
-cd src
-
-if [ -z "${ETHERPAD_PRODUCTION}" ]; then
-  log "Installing dev dependencies"
- npm ci --no-optional --omit=optional --include=dev --lockfile-version 1 || exit 1
-else
+if [ "$NODE_ENV" = "production" ]; then
   log "Installing production dependencies"
-  npm ci --no-optional --omit=optional --omit=dev --lockfile-version 1 --production || exit 1
+  npm link ep_etherpad-lite --omit=optional --omit=dev --save --package-lock=true --bin-links=false || exit 1
+else
+  log "Installing dev dependencies"
+  npm link ep_etherpad-lite --omit=optional --save --package-lock=true --bin-links=false || exit 1
 fi
 
+log "Adding symlinks for plugin backwards compatibility"
+mkdir src/node_modules -p
+ln -s ../../node_modules/async src/node_modules/async
+ln -s ../../node_modules/cheerio src/node_modules/cheerio
+ln -s ../../node_modules/express src/node_modules/express
+ln -s ../../node_modules/formidable src/node_modules/formidable
+ln -s ../../node_modules/log4js src/node_modules/log4js
+ln -s ../../node_modules/supertest src/node_modules/supertest
+
+
 # Remove all minified data to force node creating it new
 log "Clearing minified cache..."
 rm -f var/minified*

src/bin/installOnWindows.bat

@@ -9,19 +9,35 @@ cmd /C node -e "" || ( echo "Please install node.js ( https://nodejs.org )" && e
 echo _
 echo Ensure that all dependencies are up to date...  If this is the first time you have run Etherpad please be patient.
 
-mkdir node_modules
-cd /D node_modules
-mklink /D "ep_etherpad-lite" "..\src"
+echo Deleting old node_modules and src/node_modules
+del /q .\node_modules
+del /q .\src\node_modules
+echo Deleting old package.json and package-lock.json
+del /q .\package.json
+del /q .\package-lock.json
 
-cd /D "ep_etherpad-lite"
-cmd /C npm ci || exit /B 1
+cd /D src
+cmd /C npm link --bin-links=false || exit /B 1
 
-cd /D "%~dp0\..\.."
+cd ..
+
+cmd /C npm link ep_etherpad-lite --omit=optional --omit=dev --save --package-lock=true --bin-links=false || exit /B 1
 
 echo _
 echo Clearing cache...
 del /S var\minified*
 
+echo Adding symlinks for plugin backwards compatibility
+mkdir src\node_modules
+cd /D src\node_modules
+mklink /D "async" "..\..\node_modules\async"
+mklink /D "cheerio" "..\..\node_modules\cheerio"
+mklink /D "express" "..\..\node_modules\express"
+mklink /D "formidable" "..\..\node_modules\formidable"
+mklink /D "log4js" "..\..\node_modules\log4js"
+mklink /D "supertest" "..\..\node_modules\supertest"
+cd ..\..
+
 echo _
 echo Setting up settings.json...
 IF NOT EXIST settings.json (

src/bin/plugins/checkPlugin.js

@@ -356,18 +356,17 @@ const logger = log4js.getLogger('checkPlugin');
     logger.warn('Test files not found, please create tests.  https://github.com/ether/etherpad-lite/wiki/Creating-a-plugin#writing-and-running-front-end-tests-for-your-plugin');
   }
 
-  // Install dependencies so we can run ESLint. This should also create or update package-lock.json
-  // if autoFix is enabled.
-  const npmInstall = `npm install${autoFix ? '' : ' --no-package-lock'}`;
-  execSync(npmInstall, {stdio: 'inherit'});
-  // Create the ep_etherpad-lite symlink if necessary. This must be done after running `npm install`
-  // because that command nukes the symlink.
   try {
     const d = await fsp.realpath(path.join(pluginPath, 'node_modules/ep_etherpad-lite'));
     assert.equal(d, epSrcDir);
   } catch (err) {
-    execSync(`${npmInstall} --no-save ep_etherpad-lite@file:${epSrcDir}`, {stdio: 'inherit'});
+    execSync('./src/bin/installDeps.sh', {stdio: 'inherit'});
   }
+
+  // Install dependencies so we can run ESLint. This should also create or update package-lock.json
+  // if autoFix is enabled.
+  const npmInstall = `npm install${autoFix ? '' : ' --no-package-lock'}`;
+  execSync(npmInstall, {stdio: 'inherit'});
   // linting begins
   try {
     logger.info('Linting...');

src/bin/plugins/lib/README.md

@@ -11,7 +11,7 @@ TODO: Describe the plugin.
 From the Etherpad working directory, run:
 
 ```shell
-npm install --no-save --legacy-peer-deps [plugin_name]
+npm install [plugin_name]
 ```
 
 Or, install from Etherpad's `/admin/plugins` page.

src/bin/updatePlugins.sh

@@ -8,4 +8,4 @@ OUTDATED=$(npm outdated --depth=0 | awk '{print $1}' | grep '^ep_') || {
 }
 set -- ${OUTDATED}
 echo "Updating plugins: $*"
-exec npm install --no-save "$@"
+exec npm install "$@"

src/node/utils/Minify.js

@@ -163,7 +163,7 @@ const minify = async (req, res) => {
       // Go straight into node_modules
       // Avoid `require.resolve()`, since 'mustache' and 'mustache/index.js'
       // would end up resolving to logically distinct resources.
-      filename = path.join('../node_modules/', library, libraryPath);
+      filename = path.join('../../node_modules/', library, libraryPath);
     }
   }
   const [, testf] = /^plugins\/ep_etherpad-lite\/(tests\/frontend\/.*)/.exec(filename) || [];

src/package.json

@@ -53,7 +53,6 @@
     "log4js": "0.6.38",
     "measured-core": "^2.0.0",
     "mime-types": "^2.1.35",
-    "npm": "^6.14.18",
     "openapi-backend": "^5.10.0",
     "proxy-addr": "^2.0.7",
     "rate-limiter-flexible": "^3.0.0",
@@ -94,16 +93,17 @@
   },
   "engines": {
     "node": ">=16.20.1",
-    "npm": ">=6.14.0"
+    "npm": ">=8.19.4"
   },
   "repository": {
     "type": "git",
     "url": "https://github.com/ether/etherpad-lite.git"
   },
   "scripts": {
-    "lint": "eslint .",
-    "test": "mocha --timeout 120000 --recursive tests/backend/specs ../node_modules/ep_*/static/tests/backend/specs",
-    "test-container": "mocha --timeout 5000 tests/container/specs/api",
+    "lint": "../node_modules/eslint/bin/eslint.js .",
+    "test": "../node_modules/mocha/bin/_mocha --timeout 120000 --recursive tests/backend/specs ../node_modules/ep_*/static/tests/backend/specs",
+    "test-on-windows": "mocha --timeout 120000 --recursive tests/backend/specs ../node_modules/ep_*/static/tests/backend/specs",
+    "test-container": "../node_modules/mocha/bin/_mocha --timeout 5000 tests/container/specs/api",
     "dev": "bash ./bin/run.sh"
   },
   "version": "1.9.3",

src/static/js/pluginfw/installer.js

@@ -31,10 +31,7 @@ exports.uninstall = async (pluginName, cb = null) => {
   cb = wrapTaskCb(cb);
   logger.info(`Uninstalling plugin ${pluginName}...`);
   try {
-    // The --no-save flag prevents npm from creating package.json or package-lock.json.
-    // The --legacy-peer-deps flag is required to work around a bug in npm v7:
-    // https://github.com/npm/cli/issues/2199
-    await runCmd(['npm', 'uninstall', '--no-save', '--legacy-peer-deps', pluginName]);
+    await runCmd(['npm', 'uninstall', pluginName]);
   } catch (err) {
     logger.error(`Failed to uninstall plugin ${pluginName}`);
     cb(err || new Error(err));
@@ -50,10 +47,7 @@ exports.install = async (pluginName, cb = null) => {
   cb = wrapTaskCb(cb);
   logger.info(`Installing plugin ${pluginName}...`);
   try {
-    // The --no-save flag prevents npm from creating package.json or package-lock.json.
-    // The --legacy-peer-deps flag is required to work around a bug in npm v7:
-    // https://github.com/npm/cli/issues/2199
-    await runCmd(['npm', 'install', '--no-save', '--legacy-peer-deps', pluginName]);
+    await runCmd(['npm', 'install', pluginName]);
   } catch (err) {
     logger.error(`Failed to install plugin ${pluginName}`);
     cb(err || new Error(err));