docs: Expand "Extending Caddy" to new Developers section

src/docs/markdown/extending-caddy.md

@@ -4,17 +4,20 @@ title: "Extending Caddy"
 
 # Extending Caddy
 
-Caddy is easy to extend because of its modular architecture. Most kinds of Caddy extensions (or plugins) are known as _modules_ if they extend or plug into Caddy's configuration structure. To be clear, Caddy modules are distinct from [Go modules](https://github.com/golang/go/wiki/Modules) (but they are also Go modules). On this page, we refer to Caddy modules.
+Caddy is easy to extend because of its modular architecture. Most kinds of Caddy extensions (or plugins) are known as _modules_ if they extend or plug into Caddy's configuration structure. To be clear, Caddy modules are distinct from [Go modules](https://github.com/golang/go/wiki/Modules) (but they are also Go modules).
+
+**Prerequisites:**
+- Basic understanding of [Caddy's architecture](/docs/architecture)
+- Go language proficiency
+- [`go`](https://golang.org/dl)
+- [`xcaddy`](https://github.com/caddyserver/xcaddy)
 
 
+## Quick Start
 
-## Module Basics
+A Caddy module is any named type that registers itself as a Caddy module when its package is imported. Crucially, a module always implements the [caddy.Module](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#Module) interface, which provides its name and a constructor function.
 
-A Caddy module is any named type that registers itself as a Caddy module when its package is imported.
+In a new Go module, paste the following template into a Go file and customize your package name, type name, and Caddy module ID:
-
-Crucially, a module always implements the [caddy.Module](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#Module) interface, which provides its name and a constructor function.
-
-Here's a template you can copy & paste:
 
 ```go
 package mymodule
@@ -38,13 +41,27 @@ func (Gizmo) CaddyModule() caddy.ModuleInfo {
 }
 ```
 
-A module is "plugged in" by adding an import to the program:
+Then run this command from your project's directory, and you should see your module in the list:
+
+<pre><code class="cmd bash">xcaddy list-modules
+...
+foo.gizmo
+...</code></pre>
+
+<aside class="tip">
+	The <a href="https://github.com/caddyserver/xcaddy"><code>xcaddy</code> command</a> is an important part of every module developer's workflow. It compiles Caddy with your plugin, then runs it with the given arguments. It discards the temporary binary each time (similar to <code>go run</code>).
+</aside>
+
+Congratulations, your module registers with Caddy and can be used in [Caddy's config document](/docs/json/) in whatever places use modules in the same namespace.
+
+Under the hood, `xcaddy` is simply making a new Go module that requires both Caddy and your plugin (with an appropriate `replace` to use your local development version), then adds an import to ensure it is compiled in:
 
 ```go
 import _ "github.com/example/mymodule"
 ```
 
-## Module Requirements
+
+## Module Basics
 
 Caddy modules:
 
@@ -52,7 +69,10 @@ Caddy modules:
 2. Have a unique name in the proper namespace
 3. Usually satisfy some interface(s) that are meaningful to the host module for that namespace
 
-The next sections explain how to satisfy these properties!
+**Host modules** (or _parent modules_) are modules which load/initialize other modules. They typically define namespaces for guest modules.
+
+**Guest modules** (or _child modules_) are modules which get loaded or initialized. All modules are guest modules.
+
 
 ## Module IDs
 
@@ -62,32 +82,28 @@ Each Caddy module has a unique ID, consisting of a namespace and name:
 - The namespace would be `foo.bar`
 - The name would be `module_name` which must be unique in its namespace
 
-**Host modules** (or _parent modules_) are modules which load/initialize other modules. They typically define namespaces for guest modules.
-
-**Guest modules** (or _child modules_) are modules which get loaded or initialized. All modules are guest modules.
-
 Module IDs must use `snake_case` convention.
 
 ### Namespaces
 
-Namespaces are like classes, i.e. a namespace defines some functionality that is common among all modules within it. For example, we can expect that all modules within the `http.handlers` namespace are HTTP handlers. It follows that a host module may type-assert guest modules in that namespace from `interface{}` types into a more specific, useful type such as `http.Handler`.
+Namespaces are like classes, i.e. a namespace defines some functionality that is common among all modules within it. For example, we can expect that all modules within the `http.handlers` namespace are HTTP handlers. It follows that a host module may type-assert guest modules in that namespace from `interface{}` types into a more specific, useful type such as `caddyhttp.MiddlewareHandler`.
 
-A guest module must be properly namespaced in order for it to be recognized by a host module, because host modules will often ask Caddy core for a list of all modules within a certain namespace for a specific functionality it requires.
+A guest module must be properly namespaced in order for it to be recognized by a host module because host modules will ask Caddy for modules within a certain namespace to provide the functionality desired by the host module. For example, if you were to write an HTTP handler module called `gizmo`, your module's name would be `http.handlers.gizmo`, because the `http` app will look for handlers in the `http.handlers` namespace.
 
-Usually, but not always, a Caddy module namespaces correlates with a Go interface type that the modules in that namespace are expected to implement.
+Put another way, Caddy modules are expected to implement [certain interfaces](/docs/extending-caddy/namespaces) depending on their module namespace. With this convention, module developers can say intuitive things such as, "All modules in the `http.handlers` namespace are HTTP handlers." More technically, this usually means, "All modules in the `http.handlers` namespace implement the `caddyhttp.MiddlewareHandler` interface." Because that method set is known, the more specific type can be asserted and used.
 
-For example, if you were to write an HTTP handler module called `gizmo`, your module's name would be `http.handlers.gizmo`, because the `http` app will look for handlers in the `http.handlers` namespace.
+**[View a table mapping all the standard Caddy namespaces to their Go types.](/docs/extending-caddy/namespaces)**
 
-The `caddy` and `admin` namespaces are reserved.
+The `caddy` and `admin` namespaces are reserved and cannot be app names.
 
 To write modules which plug into 3rd-party host modules, consult those modules for their namespace documentation.
 
 ### Names
 
-The name within a namespace is not particularly important, as long as it is unique, concise, and makes sense for what it does.
+The name within a namespace is significant and highly visible to users, but is not particularly important, as long as it is unique, concise, and makes sense for what it does.
 
 
-### Apps
+## App Modules
 
 Apps are modules with an empty namespace, and which conventionally become their own top-level namespace. App modules implement the [caddy.App](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#App) interface.
 
@@ -141,7 +157,7 @@ Note that multiple loaded instances of your module may overlap at a given time!
 
 A module's configuration will be unmarshaled into its value automatically. This means, for example, that struct fields will be filled out for you.
 
-However, if your module requires additional provisioning steps, you can implement the [caddy.Provisioner](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#Provisioner) interface:
+However, if your module requires additional provisioning steps, you can implement the (optional) [caddy.Provisioner](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#Provisioner) interface:
 
 ```go
 // Provision sets up the module.
@@ -151,9 +167,9 @@ func (g *Gizmo) Provision(ctx caddy.Context) error {
 }
 ```
 
-This is typically where host modules will load their guest/child modules, but it can be used for pretty much anything.
+This is typically where host modules will load their guest/child modules, but it can be used for pretty much anything. Module provisioning is done in an arbitrary order.
 
-Provisioning MUST NOT depend on other apps, since provisioning apps is done in an arbitrary order. To rely on other app modules, you must wait until after the Provision phase.
+A module may access other apps by calling `ctx.App()`, but modules must not have circular dependencies. In other words, a module loaded by the `http` app cannot depend on the `tls` app if a module loaded by the `tls` app depends on the `http` app. (Very similar to rules forbidding import cycles in Go.)
 
 Additionally, you should avoid performing expensive operations in `Provision`, since provisioning is performed even if a config is only being validated. When in the provisioning phase, do not expect that the module will actually be used.
 
@@ -174,7 +190,7 @@ Then you can emit structured, leveled logs using `g.logger`. See [zap's godoc](h
 
 ### Validating
 
-Modules which would like to validate their configuration may do so by satisfying the [`caddy.Validator`](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#Validator) interface:
+Modules which would like to validate their configuration may do so by satisfying the (optional) [`caddy.Validator`](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#Validator) interface:
 
 ```go
 // Validate validates that the module has a usable config.
@@ -184,7 +200,7 @@ func (g Gizmo) Validate() error {
 }
 ```
 
-Validate should be a read-only function. It is run during the provisioning phase right after the `Provision()` method.
+Validate should be a read-only function. It is run after the `Provision()` method.
 
 
 ### Interface guards
@@ -271,6 +287,9 @@ Note that the `LoadModule()` call takes a pointer to the struct and the field na
 
 If a guest module must explicitly be set by the user, you should return an error if the Raw field is nil or empty before trying to load it.
 
+Notice how the loaded module is type-asserted: `g.Gadget = val.(Gadgeter)` - this is because the returned `val` is a `interface{}` type which is not very useful. However, we expect that all modules in the declared namespace (`foo.gizmo.gadgets` from the struct tag in our example) implement the `Gadgeter` interface, so this type assertion is safe, and then we can use it!
+
+If your host module defines a new namespace, be sure to document both that namespace and its Go type(s) for developers [like we have done here](/docs/extending-caddy/namespaces).
 
 ## Complete Example
 

src/docs/markdown/extending-caddy/caddyfile.md

@@ -0,0 +1,124 @@
+---
+title: "Caddyfile Support"
+---
+
+# Caddyfile Support
+
+Caddy modules are automatically added to the [native JSON config](/docs/json/) by virtue of their namespace when they are [registered](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#RegisterModule), making them both usable and documented. This makes Caddyfile support purely optional, but it is often requested by users who prefer the Caddyfile.
+
+## Unmarshaler
+
+To add Caddyfile support for your module, simply implement the [`caddyfile.Unmarshaler`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/caddyfile?tab=doc#Unmarshaler) interface. You get to choose the Caddyfile syntax your module has by how you parse the tokens.
+
+An unmarshaler's job is simply to set up your module's type, e.g. by populating its fields, using the [`caddyfile.Dispenser`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/caddyfile?tab=doc#Dispenser) passed to it. For example, a module type named `Gizmo` might have this method:
+
+```go
+// UnmarshalCaddyfile implements caddyfile.Unmarshaler. Syntax:
+//
+//     gizmo <name> [<option>]
+//
+func (g *Gizmo) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
+	for d.Next() {
+		if !d.Args(&g.Name) {
+			// not enough args
+			return d.ArgErr()
+		}
+		if d.NextArg() {
+			// optional arg
+			g.Option = d.Val()
+		}
+		if d.NextArg() {
+			// too many args
+			return d.ArgErr()
+		}
+	}
+	return nil
+}
+```
+
+It is a good idea to document the syntax in the godoc comment for the method. See the [godoc for the `caddyfile` package](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/caddyfile?tab=doc) for more information about parsing the Caddyfile.
+
+It is also important for an unmarshaler to accept multiple occurrences of its directive (rare, but can happen in some cases). Since the first token will typically be the module's name or directive (and can often be skipped by the unmarshaler), this usually means wrapping your parsing logic in a `for d.Next() { ... }` loop.
+
+Make sure to check for missing or excess arguments.
+
+You should also add an [interface guard](/docs/extending-caddy#interface-guards) to ensure the interface is satisfied properly:
+
+```go
+var _ caddyfile.Unmarshaler = (*Gizmo)(nil)
+```
+
+### Blocks
+
+To accept more configuration than can fit on a single line, you may wish to allow a block with subdirectives. This can be done using `d.NextBlock()` and iterating until you return to the original nesting level:
+
+```go
+for nesting := d.Nesting(); d.NextBlock(nesting); {
+	switch d.Val() {
+		case "sub_directive_1":
+			// ...
+		case "sub_directive_2":
+			// ...
+	}
+}
+```
+
+As long as each iteration of the loop consumes the entire segment (line or block), then this is an elegant way to handle blocks.
+
+## HTTP Directives
+
+The HTTP Caddyfile is Caddy's default Caddyfile adapter syntax (or "server type"). It is extensible, meaning you can [register](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile?tab=doc#RegisterDirective) your own "top-level" directives for your module:
+
+```go
+func init() {
+	httpcaddyfile.RegisterDirective("gizmo", parseCaddyfile)
+}
+```
+
+If your directive only returns a single HTTP handler (as is common), you may find [`RegisterHandlerDirective`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile?tab=doc#RegisterHandlerDirective) easier:
+
+```go
+func init() {
+	httpcaddyfile.RegisterHandlerDirective("gizmo", parseCaddyfileHandler)
+}
+```
+
+The basic idea is that [the parsing function](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile?tab=doc#UnmarshalFunc) you associate with your directive returns one or more [`ConfigValue`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile?tab=doc#ConfigValue) values. (Or, if using `RegisterHandlerDirective`, it simply returns the populated `caddyhttp.MiddlewareHandler` value directly.) Each config value is associated with a "class" which helps the HTTP Caddyfile adapter to know which part(s) of the final JSON config it can be used in. All the config values get dumped into a pile from which the adapter draws when constructing the final JSON config.
+
+This design allows your directive to return any config values for any recognized classes, which means it can influence any parts of the config that the HTTP Caddyfile adapter has a designated class for.
+
+If you've already implemented the `UnmarshalCaddyfile()` method, then your parse function could be as simple as:
+
+```go
+// parseCaddyfileHandler unmarshals tokens from h into a new middleware handler value.
+func parseCaddyfileHandler(h httpcaddyfile.Helper) (caddyhttp.MiddlewareHandler, error) {
+	var g Gizmo
+	err := g.UnmarshalCaddyfile(h.Dispenser)
+	return g, err
+}
+```
+
+See the [`httpcaddyfile` package godoc](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile?tab=doc) for more information about how to use the `httpcaddyfile.Helper` type.
+
+
+### Classes
+
+This table describes each class with exported types that is recognized by the HTTP Caddyfile adapter:
+
+Class name | Expected type | Description
+---------- | ------------- | -----------
+bind | `[]string` | Server listener bind addresses
+tls.connection_policy | `*caddytls.ConnectionPolicy` | TLS connection policy
+route | `caddyhttp.Route` | HTTP handler route
+error_route | `*caddyhttp.Subroute` | HTTP error handling route
+tls.cert_issuer | `certmagic.Issuer` | TLS certificate issuer
+tls.cert_loader | `caddytls.CertificateLoader` | TLS certificate loader
+
+
+## Server Types
+
+Structurally, the Caddyfile is a simple format, so there can be different types of Caddyfile formats (sometimes called "server types") to suit different needs.
+
+The default Caddyfile format is the HTTP Caddyfile, which you are probably familiar with. This format primarily configures the [`http` app](/docs/modules/http) while only potentially sprinkling some config in other parts of the Caddy config structure (e.g. the `tls` app to load and automate certificates).
+
+To configure apps other than HTTP, you may want to implement your own config adate

src/docs/markdown/extending-caddy/config-adapters.md

@@ -0,0 +1,62 @@
+---
+title: "Writing Config Adapters"
+---
+
+# Writing Config Adapters
+
+For various reasons, you may wish to configure Caddy using a format that is not [JSON](/docs/json/). Caddy has first-class support for this through [config adapters](/docs/config-adapters).
+
+If one does not already exist for the language/syntax/format you prefer, you can write one!
+
+## Template
+
+Here's a template you can start with:
+
+```go
+package myadapter
+
+import (
+	"fmt"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+)
+
+func init() {
+	caddyconfig.RegisterAdapter("adapter_name", MyAdapter{})
+}
+
+// MyAdapter adapts ____ to Caddy JSON.
+type MyAdapter struct{
+}
+
+// Adapt adapts the body to Caddy JSON.
+func (a MyAdapter) Adapt(body []byte, options map[string]interface{}) ([]byte, []caddyconfig.Warning, error) {
+	// TODO: parse body and convert it to JSON
+	return nil, nil, fmt.Errorf("not implemented")
+}
+```
+
+- See godoc for [`RegisterAdapter()`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig?tab=doc#RegisterAdapter)
+- See godoc for ['Adapter'](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig?tab=doc#Adapter) interface
+
+The returned JSON should **not** be indented; it should always be compact. The caller can always prettify it if they want to.
+
+Note that while config adapters are Caddy _plugins_, they are not Caddy _modules_ because they do not integrate into a part of the config (but they will show up in `list-modules` for convenience). Thus, they do not have `Provision()` or `Validate()` methods or follow the rest of the module lifecycle. They need only implement the `Adapter` interface and be registered as adapters.
+
+When populating fields of the config that are `json.RawMessage` types (i.e. module fields), use the `JSON()` and `JSONModuleObject()` functions:
+
+- [`caddyconfig.JSON()`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig?tab=doc#JSON) is for marshaling module values without the module name embedded. (Often used for ModuleMap fields where the module name is the map key.)
+- [`caddyconfig.JSONModuleObject()`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig?tab=doc#JSONModuleObject) is for marshaling module values with the module name added to the object. (Used pretty much everywhere else.)
+
+
+## Caddyfile Server Types
+
+It is also possible to implement a custom Caddyfile format. The Caddyfile adapter is a single adapter implementation and its default "server type" is HTTP, but it supports alternate "server types" at registration. For example, the HTTP Caddyfile is registered like so:
+
+```go
+func init() {
+	caddyconfig.RegisterAdapter("caddyfile",  caddyfile.Adapter{ServerType: ServerType{}})
+}
+```
+
+You would implement the [`caddyfile.ServerType` interface](https://pkg.go.dev/github.com/caddyserver/caddy/v2/caddyconfig/caddyfile?tab=doc#ServerType) and register your own adapter accordingly.

src/docs/markdown/extending-caddy/namespaces.md

@@ -0,0 +1,31 @@
+---
+title: "Module Namespaces"
+---
+
+# Module Namespaces
+
+Caddy guest modules are loaded generically as `interface{}` types. In order for the host modules to be able to use them, the loaded guest modules are usually type-asserted to a known type first. This page describes the mapping from module namespaces to Go types for all the standard modules.
+
+Documentation for non-standard module namespaces can be found with the documentation for the host module that defines them.
+
+<aside class="tip">
+	One way to read this table is, "If your module is in &lt;namespace&gt;, then it should compile as &lt;type&gt;."
+</aside>
+
+Namespace | Expected Type | Description
+--------- | ------------- | -----------
+|         | [`caddy.App`](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#App) | Caddy app
+caddy.logging.encoders.filter | [`logging.LogFieldFilter`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/logging?tab=doc#LogFieldFilter) | Log field filter</i>
+caddy.logging.writers | [`caddy.WriterOpener`](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#WriterOpener) | Log writers
+caddy.storage | [`caddy.StorageConverter`](https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#StorageConverter) | Storage backends
+http.authentication.hashes | [`caddyauth.Comparer`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddyhttp/caddyauth?tab=doc#Comparer) | Password hashers/comparers
+http.authentication.providers | [`caddyauth.Authenticator`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddyhttp/caddyauth?tab=doc#Authenticator) | HTTP authentication providers
+http.handlers | [`caddyhttp.MiddlewareHandler`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddyhttp?tab=doc#MiddlewareHandler) | HTTP handlers
+http.matchers | [`caddyhttp.RequestMatcher`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddyhttp?tab=doc#RequestMatcher) | HTTP request matchers
+http.reverse_proxy.circuit_breakers | [`reverseproxy.CircuitBreaker`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy?tab=doc#CircuitBreaker) | Reverse proxy circuit breakers
+http.reverse_proxy.selection_policies | [`reverseproxy.Selector`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy?tab=doc#Selector) | Load balancing selection policies<br><i>⚠️ Subject to change</i>
+http.reverse_proxy.transport | [`http.RoundTripper`](https://pkg.go.dev/net/http?tab=doc#RoundTripper) | HTTP reverse proxy transports
+tls.certificates | [`caddytls.CertificateLoader`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddytls?tab=doc#CertificateLoader) | TLS certificate source</i>
+tls.handshake_match | [`caddytls.ConnectionMatcher`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddytls?tab=doc#ConnectionMatcher) | TLS connection matcher</i>
+tls.issuance | [`certmagic.Issuer`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Issuer) | TLS certificate issuer<br><i>⚠️ Subject to change</i>
+tls.stek | [`caddytls.STEKProvider`](https://pkg.go.dev/github.com/caddyserver/caddy/v2/modules/caddytls?tab=doc#STEKProvider) | TLS session ticket key source</i>

src/includes/docs-nav.html

@@ -43,8 +43,17 @@
 		<li><a href="/docs/v2-upgrade">Upgrading to Caddy 2</a></li>
 		<li><a href="/docs/conventions">Conventions</a></li>
 		<li><a href="/docs/config-adapters">Config Adapters</a></li>
-		<li><a href="/docs/extending-caddy">Extending Caddy</a></li>
 		<li><a href="/docs/logging">How Logging Works</a></li>
 		<li><a href="/docs/architecture">Caddy Architecture</a></li>
+
+		<li class="heading">Developers</li>
+		<li>
+			<a href="/docs/extending-caddy">Extending Caddy</a>
+			<ul>
+				<li><a href="/docs/extending-caddy/caddyfile">Caddyfile Support</a></li>
+				<li><a href="/docs/extending-caddy/config-adapters">Config Adapters</a></li>
+			</ul>
+		</li>
+		<li><a href="/docs/extending-caddy/namespaces">Module Namespaces</a></li>
 	</ul>
 </nav>