Troubleshoot Applications
Find why requests do not reach the origin, why a rule does not act, and what the Azion API returns when it refuses a write to an Application.
Use this page when an Application does not handle requests the way you configured it, or when the Azion API refuses a rule, a setting, or another object that belongs to it. The Application’s own symptoms come first, then those of Cache, Application Accelerator, and Image Processor, the Platform Resources nested in Applications.
Requests never reach the origin
Clients send requests to the domain of a Workload, and your origin logs none of them.
Two links connect a request to an origin. The Workload’s deployment sets the Application. A Request Phase rule with Set Connector then sets the Connector, which holds the origin’s address, because an Application reaches an origin only through a Connector.
- Set the Application on the Workload’s deployment: in Azion Console, select it in the Application field of the Workload’s Deployment Settings.
- Add a rule that sends requests to the Connector: a new Application has no rules. Create one with Set Connector in Azion Console, as Applications quickstart shows, or with
set_connectorthrough the API. - Give the change time to propagate: send the request again until the origin receives it, for the times that Propagation gives.
Requests that match the rule then reach the origin through the Connector, and the origin’s responses return to the client.
The origin returns an error
Your origin answers a request with a 4xx or 5xx status, and the client receives an error page.
The Connector that a Set Connector rule sets receives the origin’s error. The error page is set on the Workload, not on the Application: its deployment sets the page beside the Application and the Firewall.
- Serve your own error page: create it as Custom Pages quickstart shows. Then, in Azion Console, select it in the Custom Page field of the Workload’s Deployment Settings.
- Serve the last stored copy instead of a
5xx: turn on stale cache in the cache setting that a Set Cache Policy rule applies to the path, withmodules.cache.stale_cache.enabledset totruein the API.
A client then receives your page for an origin error, or the expired copy of a cached object within the stale window.
A rule does not act on the requests you expect
A rule is saved, and requests you meant it to catch keep the previous behavior.
A rule acts only on the requests its criteria match, and only after the change propagates. Check three causes:
- The rule has not propagated yet: the API accepts a rule with
202andstateset topending. Send the request again before you change the rule, for as long as Propagation describes. - The criteria do not match the request: Criteria lists each variable, operator, and conditional.
- An earlier rule ended the processing: a behavior such as Deny (403 Forbidden) stops the behaviors and rules after it, as How rules run explains.
To tell the three apart, check which rules ran on the request. Debug Rules logs the Rules Engine rules that run on each request, and it is off in a new Application.
- Turn on Debug Rules: in Azion Console, on the Main Settings tab of the Application, turn on Debug Rules and select Save. Through the API, set the Application’s
debugfield totrue. - Read the rules that ran: the
$tracebackfield of each log lists them, as Debug Rules describes. A rule missing from the log of a request did not run on it: rework its criteria, or move it ahead of the rule that ended the processing.
The rule then appears in the log of each request it matches, and its behaviors run.
A problem shows in traffic but not in the responses you request
Users report errors or slow responses, yet each request you send to the Application returns what you expect.
A request you send shows one response, from one server, at one moment, as Debug headers shows. A problem that touches some servers, some paths, or some hours does not show in a single response.
- Read aggregated metrics: Real-Time Metrics charts aggregated data for your Applications, and keeps it over longer storage periods.
- Read the raw log of each request: Real-Time Events holds the raw data of every request.
- Query only the data you need: the GraphQL API returns aggregated and raw data, limited to the fields a query asks for, as Query usage data from Applications shows.
- Trace the rules that ran: Debug Rules logs them per request.
The metrics and logs then show which requests, paths, or periods carry the problem.
A rule is refused with Missing Required Modules
Saving a request rule that reads ${request_uri} fails with 400 and code 25047 while Application Accelerator is off, as Errors shows in full.
Some variables and behaviors belong to a Platform Resource, and the API refuses a rule that uses one while that Platform Resource is off: meta.missing_required_modules names it.
- Match on
${uri}instead: it needs no Platform Resource, so the same rule returns202, as Variables lists for every variable. - Turn on the Platform Resource the error names: in Azion Console, its switch is in the Modules section of the Application’s Main Settings tab, and Save applies it. Main Settings gives its Azion CLI flag and API key.
With Application Accelerator on, the rule on ${request_uri} is accepted.
A behavior is refused with Invalid Choice
A rule body copied from an older example fails with 400, code 10039, and Invalid Choice, which Errors lists with its detail and pointer.
The API accepts only the behavior types it defines. The one that adds a header to the request is add_request_header, Add Request Header in Azion Console, and not add_header.
- Send
add_request_header: carry the header asname: valueinattributes.value, as in{ "type": "add_request_header", "attributes": { "value": "Accept: image/webp" } }. - Look up the other types: Behaviors lists every behavior and its type.
The rule is then accepted and reads back with the type add_request_header.
A WebSocket connection does not return 101 Switching Protocols
A WebSocket connection through the Application returns a status other than 101 Switching Protocols, even a 2xx or a 3xx.
Only 101 means an upgraded connection: any other status means that the client, the Application, or the origin did not complete it.
- Confirm native WebSocket support: the client and the origin both need it, as Client and origin support requires.
- Send both upgrade headers:
Upgrade: websocketandConnection: upgrade, as Upgrade headers requires. - Confirm the account can use WebSocket Proxy: Availability names who has access.
The connection then returns 101 Switching Protocols.
A WebSocket connection closes after about 15 minutes
A WebSocket connection through the Application closes after about 15 minutes, while the client and the origin still use it.
Azion recycles the keepalive connections on its servers approximately every 15 minutes, and the recycle can close an active WebSocket connection, as WebSocket Proxy states.
- Reopen the connection from the client: detect the closed connection in your client code, and open a new one.
The client then holds a new connection, which returns 101 Switching Protocols.
Cache
Every new Application has Cache on, yet a cache setting changes only the requests that a rule applies it to. For how Cache answers a request, refer to the Cache overview.
A saved cache setting has no effect and responses stay MISS
x-cache reads MISS on each response for the path, and your origin gets as many requests as before.
A saved cache setting acts only on the requests that a propagated Request Phase rule applies it to through Set Cache Policy.
- Apply the setting with a rule: add Set Cache Policy with the setting to a Request Phase rule that covers the path, as Create a cache setting shows.
- Wait for the rule to propagate: Propagation gives the time it takes.
- Read the status of several requests: each one can come from a different server. Check the cache status of a response shows how to read it.
With the rule in effect, the request that stores the copy reports MISS, and later responses served from that copy report HIT.
The x-cache header reports a dash on every response for a path
Both debug headers hold a dash, x-cache: - and x-cache-key: -, so the response has neither a cache status nor a key.
The dash marks content restricted from caching, as Cache status lists. GET and HEAD responses are cached natively, while POST and OPTIONS responses need Application Accelerator and a cache setting that varies by their method.
- Turn on Application Accelerator first: with it off, the API refuses the method field with error
21013. Main Settings has the switch. - Cache the method: in Azion Console, select it in Cache vary by Method, in the Application Accelerator section of the cache setting, which then reads back as
"cache_vary_by_method":["post"]for POST. Cache POST and OPTIONS responses has the steps. - Tell the cases apart by method: on a
GETor aHEAD, the dash comes from elsewhere, such as a response that a Function builds.
Responses to the cached method then carry a status, and a key that ends with an MD5 hash of the request body.
Content is still served from cache after a Bypass Cache rule
Requests that a Bypass Cache rule matches keep getting a stored copy, and the origin does not see them.
Bypass Cache keeps Azion’s cache from storing the response, but the Tiered Cache layer is outside its reach. While a cache setting keeps Tiered Cache on, that layer goes on storing objects for the minimum TTL, and it answers before the origin does.
- Turn off Tiered Cache: in Azion Console, in the Cache section of the cache setting that serves the path. Through the API, set
modules.cache.tiered_cache.enabledtofalsein that setting. - Purge both layers, the tiered one first: purge Tiered Cache by cache key, then Cache, so the first layer cannot refill from a stale copy, as Purge cached content shows.
Requests that the rule sends to the origin then carry x-cache: BYPASS.
Responses keep an old version after you replace a file at the origin
You replaced a file at the origin, and the responses for its URL keep the old version.
A stored copy stays in cache until its TTL ends or a purge removes it, even after you replace the file at the origin, as Expiration and freshness explains.
- Purge the URL: remove the stored copy, as Purge cached content shows.
- Purge Tiered Cache first when it is on: purge the Tiered Cache layer by cache key, then Cache, as Tiered Cache explains.
- Change the object’s name: put a version in the file name, so each version is a separate object, as Change an object’s name when its content changes describes.
On its next request, the purged URL reports x-cache: MISS, and the response carries the new version.
A purge did not expire a variation of the object
The purge was accepted, yet some visitors, or some query strings, keep getting the old content.
A URL purge maps each URL to one cache key without variation, so copies that vary by cookie, device group, or image format survive it, however often you repeat it. It reaches a query string variation only when the URL lists the arguments in the order the key stores them, alphabetical when the variation has sort_enabled set to true.
- Purge by cache key: list every variation of the object, at most 50 keys in one request, each copied from the
x-cache-keyheader of its response. - Purge by wildcard: end the expression with
@@*for the variations that add the@@separator, such as cookie and device group, or with?*for query string variations.
On its next request, each purged variation reports x-cache: MISS, as Purge content that varies shows for each kind of variation.
A Max Age below 60 seconds is refused with error 21021
Creating or updating a cache setting with a Max Age below 60 seconds returns 400 and code 21021, as Errors details.
Without Application Accelerator, the floor is 60 seconds, with Tiered Cache on or off. With it, a Max Age of 30 or of 0 saves, unless Tiered Cache is on, whose floor is 3 seconds.
- Turn on Application Accelerator: in Azion Console, its switch is in Main Settings › Modules.
- Keep it off and raise the value: a Max Age of 60 seconds or more saves without it.
The cache setting is then accepted, within the bounds that Applications limits lists.
Tiered Cache cannot be turned on and the API returns 21001 or 21020
A cache setting that sends tiered_cache.enabled as true is refused with 400: code 21001 answers the honor cache behavior, and code 21020 a Max Age below 3 seconds, as Errors details for each.
Tiered Cache needs the Override cache behavior in Azion Console, override in the API, and a Max Age of at least 3 seconds. With Application Accelerator off, the 60-second floor of error 21021 applies first, so 3 to 59 seconds is still refused.
- Send the behavior and the Max Age in one request:
modules.cache.behaviorset tooverride, with amax_ageof at least 3, or of at least 60 with Application Accelerator off. - Use
--filein Azion CLI: no CLI flag sets the cache behavior, so--tiered-caching-enabledon its own always ends in21001. This command writes the body to a file and creates the setting from it:
The CLI prints Created Cache Settings configuration with ID <cache-setting-id>, and the setting reads back with enabled set to true under tiered_cache. For the topologies, refer to Tiered Cache.
A cache setting cannot be deleted and the API returns 21014
A DELETE on a cache setting answers 400 and code 21014, which Errors lists with its detail.
The Set Cache Policy behavior of some rule still applies the setting, and a setting in use cannot be removed.
- Release the setting: in Azion Console, on the Rules Engine tab of the Application, point that rule’s Set Cache Policy at a different setting, or delete the rule.
- Delete it again: with Azion CLI, the delete takes the ID of the Application and the ID of the setting:
With no rule applying the setting, the CLI prints {"message": "Caches settings configuration <cache-setting-id> was successfully deleted"}, and the setting leaves the Cache Settings list in Azion Console.
A purge request is rejected with 30001, 30003, or 30005
A purge comes back 400 rather than 201, with one of three codes that Errors lists with its detail:
| Code | Title | Cause | Fix |
|---|---|---|---|
30001 |
Invalid Purge Layer For Purge Type |
layer is tiered_cache on a purge by URL or by wildcard |
Purge by cache key to reach the Tiered Cache layer, or change layer to cache |
30003 |
Unauthorized Domain |
An item names a domain that does not belong to the account sending the purge | Fix the domain in the item |
30005 |
Invalid Purge Cachekey |
A cache key purge carries an item that is not a valid key | Copy the key from the x-cache-key header as is, with no scheme separator or spaces |
Azion CLI reports 30001 with the detail of the API, as this URL purge on the Tiered Cache layer shows:
The command prints Error: ["Invalid purge layer for purge type \"url\"."]. A purge by cache key on the same layer prints Purge carried out successfully, and accepted purges return 201 with state set to executed, as Request body shows.
Azion CLI prints an empty error object
A cache setting create or update fails, and under --format json Azion CLI prints only {"error": {}}.
With JSON output, Azion CLI 4.23.0 loses the code and the title that the API answered with.
- Repeat the command without
--format json: plain output carries the message of the API, such asError: Failed to create the Cache Settings configuration: ["It's not possible to use this edge cache behavior while using Tiered Cache."]for Tiered Cache withhonor. The message keeps an earlier name for Cache. - Look the message up: on this page, or in Errors, which lists every code of the cache settings endpoint.
Plain output then shows why the API refused the request, and the corrected request saves.
Application Accelerator
Application Accelerator decides what goes into a cache key, which HTTP methods get cached, how low the Max Age goes, and which Rules Engine behaviors and variables a rule may use. It is off in a new Application. For how a variation changes the key, refer to the Application Accelerator overview.
A POST request reports no cache status
A POST reports - in x-cache until Application Accelerator is on and the cache setting caches the method: POST in Cache vary by Method in Azion Console, or post in cache_vary_by_method in the API. HTTP methods shows the field for each interface. For the fix, refer to The x-cache header reports a dash on every response for a path.
Cookie and device group variations survive a purge
These variations come from the modules.application_accelerator object of the cache setting, the Application Accelerator section in Azion Console. Each one is an object under its own key, out of reach of a URL purge. For the purge by cache key or by the @@* wildcard that does, refer to A purge did not expire a variation of the object.
A URL purge misses a query string variation
A URL purge reaches a query string variation only when it lists the arguments the key varies by, in the order the key stores them. That order is alphabetical when Sort is on in Cache vary by Query String in Azion Console, sort_enabled in the API. For that purge and the ?* wildcard, refer to A purge did not expire a variation of the object.
A behavior or a variable cannot be selected in Rules Engine
In Azion Console, on the Rules Engine tab of an Application, a behavior such as Bypass Cache, Forward Cookies, or Rewrite Request cannot be added to a rule. In the list, its name ends with - Required Application Accelerator, and the ${device_group} variable is missing from the criteria.
Six behaviors and the ${device_group} variable stay locked while Application Accelerator is off in the Application.
- Turn on Application Accelerator: in Azion Console, its switch is in Main Settings › Modules.
- Turn on Functions for Run Function: Run Function depends on Functions turned on in the Application, not on Application Accelerator, as Run Function is missing from the behaviors list explains.
A rule can then take the six behaviors, and its criteria can use ${device_group}, as Rules Engine behaviors lists by phase.
Content is served from cache with Bypass Cache configured
Bypass Cache needs Application Accelerator, and a rule that applies it can still be answered from the Tiered Cache layer, which Bypass Cache leaves untouched. The rule works, so changing it fixes nothing: turn off Tiered Cache and purge both layers, as Content is still served from cache after a Bypass Cache rule describes.
A cache variation is refused with error 21013
Saving a cache setting returns 400 and code 21013 when it sets a field of its Application Accelerator section in Azion Console, or of modules.application_accelerator in the API. Errors details the code.
Those fields vary the cache by method, query string, cookie, and device group, and each one needs Application Accelerator.
- Turn on Application Accelerator, then save the setting again: in Azion Console, the switch is in Main Settings › Modules.
- Read the reason when Azion CLI hides it: with
--format json, Azion CLI 4.23.0 prints this refusal as{"error": {}}for a setting with a query string allowlist. To read the reason, refer to Azion CLI prints an empty error object.
The setting then saves, and its variation reads back as sent.
A query string variation is refused with error 21018
A cache setting whose query string behavior is allowlist or denylist, sent with an empty list of fields, returns 400 and code 21018, as Errors details.
allowlist names the fields the key varies by, and denylist names the ones it leaves out. Each needs at least one, and Azion Console makes Fields required.
- List the fields: in Azion Console, enter at least one query string parameter in Fields, one per line, or send them in
fieldsthrough the API. - Keep the default when the key should not vary: the default behavior,
ignorein the API and Ignore in Azion Console, saves with an emptyfieldslist.
The setting then saves, with cache_vary_by_querystring holding what you sent, as Cache variation lists.
Image Processor
Image Processor builds a derived image from the source image when a rule applies Optimize Images, and it is off in a new Application. For the path from the source image to the derived image, refer to the Image Processor overview.
A 504 error on a request whose ims is not the last parameter
A request for an image returns a 504 error, and its URL has another query string parameter after ims=.
ims= has to come last among the parameters of the URL: a parameter after it may make the request return a 504, though not every such request fails.
- Put
imslast: move the other parameters ahead ofims=, as inexample.com/image.jpeg?ts=1234&ims=1000x1000. For the rule, refer to Position of the ims parameter.
With ims= last, the URL returns the derived image, not a 504.
The image is delivered unprocessed and its ims query string is ignored
The ims query string of a request is valid, yet the source image comes back as it is, without a resize, a crop, or a filter.
A transformation starts only when a rule matches the request and applies Optimize Images, whatever the ims query string asks for.
- Check what the rule matches: in Azion Console, on the Rules Engine tab of the Application, compare the rule’s criterion with the request. A criterion for images compares
${uri}, or${request_uri}with Application Accelerator on, throughmatcheswith an argument such as\.(jpg|jpeg|gif|bmp|png|ico|webp|avif). - Check the behavior: Optimize Images has to be a Request Phase behavior of the rule. When the rule also caches the result, Set Cache Policy sits beside it, as Rules Engine behaviors describes.
- Follow the guide: Configure Image Processor on an application builds the rule step by step.
A request that matches the rule then gets a derived image, with x-ims: Enabled in the response headers.
The image is not converted to WEBP or AVIF
You request a WEBP or AVIF conversion, and the image arrives in another format.
A conversion happens only for a request whose Accept header names the format, as Format conversion requires. WEBP needs Accept: image/webp, and AVIF needs Accept: image/avif.
- Set the header from a rule: give the rule an Add Request Header behavior in its Request Phase,
add_request_headerin the API. Its value isAccept: image/webporAccept: image/avif, to match the format theimsstring requests. For the behavior, refer to Image Processor settings.
Matching requests then get the image back as image/webp or image/avif, the format the ims string requests.
Every request for the same derived image is processed again
Each request for the same image with the same ims value adds a processed image to the Images meter of the account, instead of coming from cache.
Optimize Images processes the image, and keeping the result takes a cache setting that a rule applies with Set Cache Policy.
- Apply a cache setting to the image requests: add Set Cache Policy, with the cache setting that serves images, to the rule that applies Optimize Images. For the query string allowlist that this setting carries, refer to Cache variation.
The rule then applies the cache setting to each image request it matches, and a request that the cache answers adds no processed image.
A resize larger than the original does not enlarge the image
You ask for fit-in/WidthxHeight, or for dimensions beyond those of the source image, and the derived image comes back no bigger than the original.
The resize stops at the size of the source image, in every configuration. fit-in scales an image down only, and without fit-in, a request for more resolution than the original has returns the highest resolution the source allows.
The result keeps the size of the source image, or takes the largest size that fits the requested box, as Resize shows for every form.
A rotation value has no effect
With ?ims=filters:rotate(Angle) set to an angle other than 0, 90, 180, 270, or 360, the derived image keeps its orientation.
Rotation accepts only those five angles, and any other value leaves the image unrotated.
- Pick an accepted angle: Rotation gives the syntax of the filter.
The derived image is then turned to the left by the angle you requested.