Encoding in Web Applications
Web applications move data constantly—through URLs, form fields, cookies, headers, and API payloads. The catch: many transport mechanisms (especially URLs and HTML) are text-oriented, while real input often includes spaces, symbols, non‑English characters, and even binary bytes.
That’s why multiple encoding schemes exist. They help data travel safely—but they can also introduce surprising behavior when systems decode at different stages. If you build or test web apps, understanding encoding is non‑negotiable.
Common Encoding Reference
This table shows how the most important characters (especially the ones used in HTML “tags”) appear in different encodings.
Notes:
Use HTML entities when you want user input to be shown safely on a web page.
Use URL encoding when you put data inside a URL or query parameter.
Numeric HTML encodings are valid alternatives to named entities.
1) URL Encoding (Percent-Encoding)
Why it exists
A URL can’t safely carry every character as-is. Some characters are reserved because they control URL structure:
? starts the query string
& separates parameters
=
# starts the fragment
% signals encoded bytes
If these appear inside data (like a username or search term), they should be encoded so they aren’t misread as URL syntax.
How it works
URL encoding replaces a character with:
% + two-digit hexadecimal byte value
Examples (common in requests):
= → %3D
%→%25space →
%20newline →
%0Anull byte →
%00
+ is a special case
In many query strings (application/x-www-form-urlencoded), + means space.
Example:
Raw text: coffee mug
Encoded query value: coffee+mug or coffee%20mug
Example :-
Suppose you want to send this value in a query parameter:
note = "Plan A & Plan B = approved?"
Correct URL-encoded value:
Plan%20A%20%26%20Plan%20B%20%3D%20approved%3F
So the URL becomes:
/search?note=Plan%20A%20%26%20Plan%20B%20%3D%20approved%3F
Practical tip
When you insert data into a parameter value, you typically encode these characters to avoid breaking parsing:
space % ? & = ; + #
(But when you intentionally use them as separators—like adding a new parameter—you keep them literal.)
2) Unicode & UTF‑8 Encoding
Why it matters
Unicode lets software represent almost every writing system and symbol. But different layers of a web stack may interpret Unicode differently, creating validation gaps.
UTF‑8 (the web default)
UTF‑8 encodes characters into 1–4 bytes. When bytes must be placed in a URL, each byte can be percent-encoded.
Examples:
₹ may appear as: %E2%82%B9
✓ may appear as: %E2%9C%93
%uXXXX style (legacy/edge)
Some older systems use a “Unicode escape” form like %u2215 for certain symbols. Not all servers support it consistently.
Security relevance
If a filter blocks a character (say / or <) but the next component decodes Unicode/UTF‑8 later, an attacker might sneak the dangerous character through in an encoded form. This is why “what decodes where” is crucial.
3) HTML Encoding (Entity Encoding)
Why it exists
HTML uses characters like <, >, and & as syntax, not content. If user input is inserted into HTML without encoding, it may be interpreted as markup or script.
Standard entities
< → <
> → >
& → &
"→ "'→ '(commonly used)
Numeric encoding (decimal/hex)
Any character can be encoded using its code point:
"can be:"(decimal) or"(hex)
Example:-
If a user submits:
Hello <b>world</b> & welcome!
To display it as text (not bold), output should become:
Hello <b>world</b> & welcome!
Why testers care (XSS insight)
If the app reflects user input without encoding, it may be vulnerable. If special characters are encoded correctly, risk reduces—but context matters (HTML, attribute, JS string, URL, CSS each has different rules).
4) Base64 Encoding
What it is (and what it is NOT)
Base64 is an encoding that turns bytes into printable ASCII. It is not encryption—it’s easily reversible.
Where you see it
Basic HTTP auth headers
Tokens in cookies/local storage
API payload “blobs”
“Obfuscated” values in parameters
How it works
It converts data in 3-byte blocks into 4 characters using this set:
A–Z a–z 0–9 + /
Padding: ends with = or == when needed.
Example:-
Plain text:
CloudNotes:trial_user=raj
Base64-encoded:
Q2xvdWROb3Rlczp0cmlhbF91c2VyPWFubnU=
Practical tip
If a cookie looks like Base64, try:
decoding it to see if it becomes readable JSON/text
re-encoding after small changes and testing whether the server validates integrity (many systems sign tokens—changes will break)
5) Hex Encoding
What it is
Hex encoding represents bytes using characters 0–9 and a–f. It’s common for:
IDs and hashes
binary-to-text transport
debug and logging formats
Example:-
Text: sun
Hex output:
73756e
Practical tip
If you see a value that’s only [0-9a-f] and has even length, it might be hex. Decode it and check whether it’s text, a serialized structure, or raw bytes.
6) Remoting & Serialization Frameworks
Modern apps often call server-side functions through structured APIs. Frameworks handle:
remote procedure calls
data serialization/deserialization
Examples (general categories):
binary messaging formats
platform-specific object serialization
RPC layers that hide HTTP details
Why it matters
Serialization formats can create security issues if:
untrusted input is deserialized into objects
type handling is unsafe
integrity checks are missing
decoding happens multiple times in different layers
Quick Builder’s Checklist
URLs: encode parameter values properly; don’t manually concatenate raw strings.
HTML output: context-encode user input (HTML/attribute/JS/URL/CSS contexts differ).
Tokens: don’t rely on Base64 to “hide” secrets—use encryption/signatures.
Validation: define where decoding happens and ensure filters run on the final interpreted form.
Logging: log both raw and decoded forms carefully (avoid leaking secrets).
Thanks for reading! ❤️❤️❤️
Please take a moment to answer the poll below and share your feedback.
please give any suggestion