Filename | /Users/ether/.perlbrew/libs/36.0@std/lib/perl5/darwin-2level/Cpanel/JSON/XS.pm |
Statements | Executed 21 statements in 4.89ms |
Calls | P | F | Exclusive Time |
Inclusive Time |
Subroutine |
---|---|---|---|---|---|
15 | 2 | 1 | 6.28ms | 6.28ms | encode (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 2.04ms | 2.32ms | BEGIN@2334 | Cpanel::JSON::XS::
1 | 1 | 1 | 1.44ms | 2.91ms | BEGIN@115 | DynaLoader::
14 | 2 | 2 | 1.41ms | 1.41ms | decode (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 480µs | 480µs | DESTROY (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 29µs | 53µs | BEGIN@235 | Cpanel::JSON::XS::
1 | 1 | 1 | 19µs | 19µs | END (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 13µs | 14µs | BEGIN@236 | Cpanel::JSON::XS::
1 | 1 | 1 | 9µs | 9µs | new (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 8µs | 8µs | BEGIN@2387 | Cpanel::JSON::XS::
1 | 1 | 1 | 7µs | 7µs | BEGIN@2375 | Cpanel::JSON::XS::
1 | 1 | 1 | 1µs | 1µs | __ANON__ (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 1µs | 1µs | allow_nonref (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 1µs | 1µs | canonical (xsub) | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | __ANON__[:2390] | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | __ANON__[:2391] | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | allow_bigint | Cpanel::JSON::XS::
1 | 1 | 1 | 0s | 0s | allow_bignum (xsub) | Cpanel::JSON::XS::
1 | 1 | 1 | 0s | 0s | convert_blessed (xsub) | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | from_json | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | is_bool | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | to_json | Cpanel::JSON::XS::
1 | 1 | 1 | 0s | 0s | utf8 (xsub) | Cpanel::JSON::XS::
0 | 0 | 0 | 0s | 0s | __ANON__[:2344] | JSON::PP::Boolean::
0 | 0 | 0 | 0s | 0s | __ANON__[:2345] | JSON::PP::Boolean::
0 | 0 | 0 | 0s | 0s | __ANON__[:2346] | JSON::PP::Boolean::
0 | 0 | 0 | 0s | 0s | __ANON__[:2351] | JSON::PP::Boolean::
0 | 0 | 0 | 0s | 0s | __ANON__[:2365] | JSON::PP::Boolean::
0 | 0 | 0 | 0s | 0s | __ANON__[:2370] | JSON::PP::Boolean::
Line | State ments |
Time on line |
Calls | Time in subs |
Code |
---|---|---|---|---|---|
0 | 1 | 2.91ms | Profile data that couldn't be associated with a specific line: # spent 2.91ms making 1 call to DynaLoader::BEGIN@115 | ||
1 | 1 | 70µs | package Cpanel::JSON::XS; | ||
2 | 1 | 1µs | our $VERSION = '4.32'; | ||
3 | 1 | 0s | our $XS_VERSION = $VERSION; | ||
4 | # $VERSION = eval $VERSION; | ||||
5 | |||||
6 | =pod | ||||
7 | |||||
8 | =head1 NAME | ||||
9 | |||||
10 | Cpanel::JSON::XS - cPanel fork of JSON::XS, fast and correct serializing | ||||
11 | |||||
12 | =head1 SYNOPSIS | ||||
13 | |||||
14 | use Cpanel::JSON::XS; | ||||
15 | |||||
16 | # exported functions, they croak on error | ||||
17 | # and expect/generate UTF-8 | ||||
18 | |||||
19 | $utf8_encoded_json_text = encode_json $perl_hash_or_arrayref; | ||||
20 | $perl_hash_or_arrayref = decode_json $utf8_encoded_json_text; | ||||
21 | |||||
22 | # OO-interface | ||||
23 | |||||
24 | $coder = Cpanel::JSON::XS->new->ascii->pretty->allow_nonref; | ||||
25 | $pretty_printed_unencoded = $coder->encode ($perl_scalar); | ||||
26 | $perl_scalar = $coder->decode ($unicode_json_text); | ||||
27 | |||||
28 | # Note that 5.6 misses most smart utf8 and encoding functionalities | ||||
29 | # of newer releases. | ||||
30 | |||||
31 | # Note that L<JSON::MaybeXS> will automatically use Cpanel::JSON::XS | ||||
32 | # if available, at virtually no speed overhead either, so you should | ||||
33 | # be able to just: | ||||
34 | |||||
35 | use JSON::MaybeXS; | ||||
36 | |||||
37 | # and do the same things, except that you have a pure-perl fallback now. | ||||
38 | |||||
39 | Note that this module will be replaced by a new JSON::Safe module soon, | ||||
40 | with the same API just guaranteed safe defaults. | ||||
41 | |||||
42 | =head1 DESCRIPTION | ||||
43 | |||||
44 | This module converts Perl data structures to JSON and vice versa. Its | ||||
45 | primary goal is to be I<correct> and its secondary goal is to be | ||||
46 | I<fast>. To reach the latter goal it was written in C. | ||||
47 | |||||
48 | As this is the n-th-something JSON module on CPAN, what was the reason | ||||
49 | to write yet another JSON module? While it seems there are many JSON | ||||
50 | modules, none of them correctly handle all corner cases, and in most cases | ||||
51 | their maintainers are unresponsive, gone missing, or not listening to bug | ||||
52 | reports for other reasons. | ||||
53 | |||||
54 | See below for the cPanel fork. | ||||
55 | |||||
56 | See MAPPING, below, on how Cpanel::JSON::XS maps perl values to JSON | ||||
57 | values and vice versa. | ||||
58 | |||||
59 | =head2 FEATURES | ||||
60 | |||||
61 | =over 4 | ||||
62 | |||||
63 | =item * correct Unicode handling | ||||
64 | |||||
65 | This module knows how to handle Unicode with Perl version higher than 5.8.5, | ||||
66 | documents how and when it does so, and even documents what "correct" means. | ||||
67 | |||||
68 | =item * round-trip integrity | ||||
69 | |||||
70 | When you serialize a perl data structure using only data types supported | ||||
71 | by JSON and Perl, the deserialized data structure is identical on the Perl | ||||
72 | level. (e.g. the string "2.0" doesn't suddenly become "2" just because | ||||
73 | it looks like a number). There I<are> minor exceptions to this, read the | ||||
74 | MAPPING section below to learn about those. | ||||
75 | |||||
76 | =item * strict checking of JSON correctness | ||||
77 | |||||
78 | There is no guessing, no generating of illegal JSON texts by default, | ||||
79 | and only JSON is accepted as input by default. the latter is a security | ||||
80 | feature. | ||||
81 | |||||
82 | =item * fast | ||||
83 | |||||
84 | Compared to other JSON modules and other serializers such as Storable, | ||||
85 | this module usually compares favourably in terms of speed, too. | ||||
86 | |||||
87 | =item * simple to use | ||||
88 | |||||
89 | This module has both a simple functional interface as well as an object | ||||
90 | oriented interface. | ||||
91 | |||||
92 | =item * reasonably versatile output formats | ||||
93 | |||||
94 | You can choose between the most compact guaranteed-single-line format | ||||
95 | possible (nice for simple line-based protocols), a pure-ASCII format | ||||
96 | (for when your transport is not 8-bit clean, still supports the whole | ||||
97 | Unicode range), or a pretty-printed format (for when you want to read that | ||||
98 | stuff). Or you can combine those features in whatever way you like. | ||||
99 | |||||
100 | =back | ||||
101 | |||||
102 | =head2 cPanel fork | ||||
103 | |||||
104 | Since the original author MLEHMANN has no public | ||||
105 | bugtracker, this cPanel fork sits now on github. | ||||
106 | |||||
107 | src repo: L<https://github.com/rurban/Cpanel-JSON-XS> | ||||
108 | original: L<http://cvs.schmorp.de/JSON-XS/> | ||||
109 | |||||
110 | RT: L<https://github.com/rurban/Cpanel-JSON-XS/issues> | ||||
111 | or L<https://rt.cpan.org/Public/Dist/Display.html?Queue=Cpanel-JSON-XS> | ||||
112 | |||||
113 | B<Changes to JSON::XS> | ||||
114 | |||||
115 | # spent 2.91ms (1.44+1.47) within DynaLoader::BEGIN@115 which was called:
# once (1.44ms+1.47ms) by XSLoader::load at line 0 | ||||
116 | safe by default. | ||||
117 | added a 2nd optional argument. decode() honors now allow_nonref. | ||||
118 | |||||
119 | - fixed encode of numbers for dual-vars. Different string | ||||
120 | representations are preserved, but numbers with temporary strings | ||||
121 | which represent the same number are here treated as numbers, not | ||||
122 | strings. Cpanel::JSON::XS is a bit slower, but preserves numeric | ||||
123 | types better. | ||||
124 | |||||
125 | - numbers ending with .0 stray numbers, are not converted to | ||||
126 | integers. [#63] dual-vars which are represented as number not | ||||
127 | integer (42+"bar" != 5.8.9) are now encoded as number (=> 42.0) | ||||
128 | because internally it's now a NOK type. However !!1 which is | ||||
129 | wrongly encoded in 5.8 as "1"/1.0 is still represented as integer. | ||||
130 | |||||
131 | - different handling of inf/nan. Default now to null, optionally with | ||||
132 | stringify_infnan() to "inf"/"nan". [#28, #32] | ||||
133 | |||||
134 | - added C<binary> extension, non-JSON and non JSON parsable, allows | ||||
135 | C<\xNN> and C<\NNN> sequences. | ||||
136 | |||||
137 | - 5.6.2 support; sacrificing some utf8 features (assuming bytes | ||||
138 | all-over), no multi-byte unicode characters with 5.6. | ||||
139 | |||||
140 | - interop for true/false overloading. JSON::XS, JSON::PP and Mojo::JSON | ||||
141 | representations for booleans are accepted and JSON::XS accepts | ||||
142 | Cpanel::JSON::XS booleans [#13, #37] | ||||
143 | Fixed overloading of booleans. Cpanel::JSON::XS::true stringifies again | ||||
144 | to "1", not "true", analog to all other JSON modules. | ||||
145 | |||||
146 | - native boolean mapping of yes and no to true and false, as in YAML::XS. | ||||
147 | In perl C<!0> is yes, C<!1> is no. | ||||
148 | The JSON value true maps to 1, false maps to 0. [#39] | ||||
149 | |||||
150 | - support arbitrary stringification with encode, with convert_blessed | ||||
151 | and allow_blessed. | ||||
152 | |||||
153 | - ithread support. Cpanel::JSON::XS is thread-safe, JSON::XS not | ||||
154 | |||||
155 | - is_bool can be called as method, JSON::XS::is_bool not. | ||||
156 | |||||
157 | - performance optimizations for threaded Perls | ||||
158 | |||||
159 | - relaxed mode, allowing many popular extensions | ||||
160 | |||||
161 | - protect our magic object from corruption by wrong or missing external | ||||
162 | methods, like FREEZE/THAW or serialization with other methods. | ||||
163 | |||||
164 | - additional fixes for: | ||||
165 | |||||
166 | - [cpan #88061] AIX atof without USE_LONG_DOUBLE | ||||
167 | |||||
168 | - #10 unshare_hek crash | ||||
169 | |||||
170 | - #7, #29 avoid re-blessing where possible. It fails in JSON::XS for | ||||
171 | READONLY values, i.e. restricted hashes. | ||||
172 | |||||
173 | - #41 overloading of booleans, use the object not the reference. | ||||
174 | |||||
175 | - #62 -Dusequadmath conversion and no SEGV. | ||||
176 | |||||
177 | - #72 parsing of values followed \0, like 1\0 does fail. | ||||
178 | |||||
179 | - #72 parsing of illegal unicode or non-unicode characters. | ||||
180 | |||||
181 | - #96 locale-insensitive numeric conversion. | ||||
182 | |||||
183 | - #154 numeric conversion fixed since 5.22, using the same strtold as perl5. | ||||
184 | |||||
185 | - #167 sort tied hashes with canonical. | ||||
186 | |||||
187 | - public maintenance and bugtracker | ||||
188 | |||||
189 | - use ppport.h, sanify XS.xs comment styles, harness C coding style | ||||
190 | |||||
191 | - common::sense is optional. When available it is not used in the | ||||
192 | published production module, just during development and testing. | ||||
193 | |||||
194 | - extended testsuite, passes all http://seriot.ch/parsing_json.html | ||||
195 | tests. In fact it is the only know JSON decoder which does so, | ||||
196 | while also being the fastest. | ||||
197 | |||||
198 | - support many more options and methods from JSON::PP: | ||||
199 | stringify_infnan, allow_unknown, allow_stringify, allow_barekey, | ||||
200 | encode_stringify, allow_bignum, allow_singlequote, dupkeys_as_arrayref, | ||||
201 | sort_by (partially), escape_slash, convert_blessed, ... | ||||
202 | optional decode_json(, allow_nonref) arg. | ||||
203 | relaxed implements allow_dupkeys. | ||||
204 | |||||
205 | - support all 5 unicode L<BOM|/BOM>'s: UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, | ||||
206 | UTF-32BE, encoding internally to UTF-8. | ||||
207 | |||||
208 | =cut | ||||
209 | |||||
210 | 1 | 9µs | our @ISA = qw(Exporter); | ||
211 | 1 | 1µs | our @EXPORT = qw(encode_json decode_json to_json from_json); | ||
212 | |||||
213 | sub to_json($@) { | ||||
214 | if ($] >= 5.008) { | ||||
215 | require Carp; | ||||
216 | Carp::croak ("Cpanel::JSON::XS::to_json has been renamed to encode_json,". | ||||
217 | " either downgrade to pre-2.0 versions of Cpanel::JSON::XS or". | ||||
218 | " rename the call"); | ||||
219 | } else { | ||||
220 | _to_json(@_); | ||||
221 | } | ||||
222 | } | ||||
223 | |||||
224 | sub from_json($@) { | ||||
225 | if ($] >= 5.008) { | ||||
226 | require Carp; | ||||
227 | Carp::croak ("Cpanel::JSON::XS::from_json has been renamed to decode_json,". | ||||
228 | " either downgrade to pre-2.0 versions of Cpanel::JSON::XS or". | ||||
229 | " rename the call"); | ||||
230 | } else { | ||||
231 | _from_json(@_); | ||||
232 | } | ||||
233 | } | ||||
234 | |||||
235 | 2 | 26µs | 2 | 77µs | # spent 53µs (29+24) within Cpanel::JSON::XS::BEGIN@235 which was called:
# once (29µs+24µs) by JSON::MaybeXS::_choose_json_module at line 235 # spent 53µs making 1 call to Cpanel::JSON::XS::BEGIN@235
# spent 24µs making 1 call to Exporter::import |
236 | 2 | 2.22ms | 2 | 15µs | # spent 14µs (13+1) within Cpanel::JSON::XS::BEGIN@236 which was called:
# once (13µs+1µs) by JSON::MaybeXS::_choose_json_module at line 236 # spent 14µs making 1 call to Cpanel::JSON::XS::BEGIN@236
# spent 1µs making 1 call to Cpanel::JSON::XS::__ANON__ |
237 | |||||
238 | =head1 FUNCTIONAL INTERFACE | ||||
239 | |||||
240 | The following convenience methods are provided by this module. They are | ||||
241 | exported by default: | ||||
242 | |||||
243 | =over 4 | ||||
244 | |||||
245 | =item $json_text = encode_json $perl_scalar, [json_type] | ||||
246 | |||||
247 | Converts the given Perl data structure to a UTF-8 encoded, binary string | ||||
248 | (that is, the string contains octets only). Croaks on error. | ||||
249 | |||||
250 | This function call is functionally identical to: | ||||
251 | |||||
252 | $json_text = Cpanel::JSON::XS->new->utf8->encode ($perl_scalar, $json_type) | ||||
253 | |||||
254 | Except being faster. | ||||
255 | |||||
256 | For the type argument see L<Cpanel::JSON::XS::Type>. | ||||
257 | |||||
258 | =item $perl_scalar = decode_json $json_text [, $allow_nonref [, my $json_type ] ] | ||||
259 | |||||
260 | The opposite of C<encode_json>: expects an UTF-8 (binary) string of an | ||||
261 | json reference and tries to parse that as an UTF-8 encoded JSON text, | ||||
262 | returning the resulting reference. Croaks on error. | ||||
263 | |||||
264 | This function call is functionally identical to: | ||||
265 | |||||
266 | $perl_scalar = Cpanel::JSON::XS->new->utf8->decode ($json_text, $json_type) | ||||
267 | |||||
268 | except being faster. | ||||
269 | |||||
270 | Note that older decode_json versions in Cpanel::JSON::XS older than | ||||
271 | 3.0116 and JSON::XS did not set allow_nonref but allowed them due to a | ||||
272 | bug in the decoder. | ||||
273 | |||||
274 | If the new 2nd optional $allow_nonref argument is set and not false, the | ||||
275 | C<allow_nonref> option will be set and the function will act is described | ||||
276 | as in the relaxed RFC 7159 allowing all values such as objects, | ||||
277 | arrays, strings, numbers, "null", "true", and "false". | ||||
278 | See L</"OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)> below, why you don't | ||||
279 | want to do that. | ||||
280 | |||||
281 | For the 3rd optional type argument see L<Cpanel::JSON::XS::Type>. | ||||
282 | |||||
283 | =item $is_boolean = Cpanel::JSON::XS::is_bool $scalar | ||||
284 | |||||
285 | Returns true if the passed scalar represents either C<JSON::PP::true> | ||||
286 | or C<JSON::PP::false>, two constants that act like C<1> and C<0>, | ||||
287 | respectively and are used to represent JSON C<true> and C<false> | ||||
288 | values in Perl. (Also recognizes the booleans produced by L<JSON::XS>.) | ||||
289 | |||||
290 | See MAPPING, below, for more information on how JSON values are mapped | ||||
291 | to Perl. | ||||
292 | |||||
293 | =back | ||||
294 | |||||
295 | =head1 DEPRECATED FUNCTIONS | ||||
296 | |||||
297 | =over | ||||
298 | |||||
299 | =item from_json | ||||
300 | |||||
301 | from_json has been renamed to decode_json | ||||
302 | |||||
303 | =item to_json | ||||
304 | |||||
305 | to_json has been renamed to encode_json | ||||
306 | |||||
307 | =back | ||||
308 | |||||
309 | |||||
310 | =head1 A FEW NOTES ON UNICODE AND PERL | ||||
311 | |||||
312 | Since this often leads to confusion, here are a few very clear words on | ||||
313 | how Unicode works in Perl, modulo bugs. | ||||
314 | |||||
315 | =over 4 | ||||
316 | |||||
317 | =item 1. Perl strings can store characters with ordinal values > 255. | ||||
318 | |||||
319 | This enables you to store Unicode characters as single characters in a | ||||
320 | Perl string - very natural. | ||||
321 | |||||
322 | =item 2. Perl does I<not> associate an encoding with your strings. | ||||
323 | |||||
324 | ... until you force it to, e.g. when matching it against a regex, or | ||||
325 | printing the scalar to a file, in which case Perl either interprets | ||||
326 | your string as locale-encoded text, octets/binary, or as Unicode, | ||||
327 | depending on various settings. In no case is an encoding stored | ||||
328 | together with your data, it is I<use> that decides encoding, not any | ||||
329 | magical meta data. | ||||
330 | |||||
331 | =item 3. The internal utf-8 flag has no meaning with regards to the | ||||
332 | encoding of your string. | ||||
333 | |||||
334 | =item 4. A "Unicode String" is simply a string where each character | ||||
335 | can be validly interpreted as a Unicode code point. | ||||
336 | |||||
337 | If you have UTF-8 encoded data, it is no longer a Unicode string, but | ||||
338 | a Unicode string encoded in UTF-8, giving you a binary string. | ||||
339 | |||||
340 | =item 5. A string containing "high" (> 255) character values is I<not> | ||||
341 | a UTF-8 string. | ||||
342 | |||||
343 | =item 6. Unicode noncharacters only warn, as in core. | ||||
344 | |||||
345 | The 66 Unicode noncharacters U+FDD0..U+FDEF, and U+*FFFE, U+*FFFF just | ||||
346 | warn, see L<http://www.unicode.org/versions/corrigendum9.html>. But | ||||
347 | illegal surrogate pairs fail to parse. | ||||
348 | |||||
349 | =item 7. Raw non-Unicode characters above U+10FFFF are disallowed. | ||||
350 | |||||
351 | Raw non-Unicode characters outside the valid unicode range fail to | ||||
352 | parse, because "A string is a sequence of zero or more Unicode | ||||
353 | characters" RFC 7159 section 1 and "JSON text SHALL be encoded in | ||||
354 | Unicode RFC 7159 section 8.1. We use now the UTF8_DISALLOW_SUPER | ||||
355 | flag when parsing unicode. | ||||
356 | |||||
357 | =back | ||||
358 | |||||
359 | I hope this helps :) | ||||
360 | |||||
361 | |||||
362 | =head1 OBJECT-ORIENTED INTERFACE | ||||
363 | |||||
364 | The object oriented interface lets you configure your own encoding or | ||||
365 | decoding style, within the limits of supported formats. | ||||
366 | |||||
367 | =over 4 | ||||
368 | |||||
369 | =item $json = new Cpanel::JSON::XS | ||||
370 | |||||
371 | Creates a new JSON object that can be used to de/encode JSON | ||||
372 | strings. All boolean flags described below are by default I<disabled>. | ||||
373 | |||||
374 | The mutators for flags all return the JSON object again and thus calls can | ||||
375 | be chained: | ||||
376 | |||||
377 | my $json = Cpanel::JSON::XS->new->utf8->space_after->encode ({a => [1,2]}) | ||||
378 | => {"a": [1, 2]} | ||||
379 | |||||
380 | =item $json = $json->ascii ([$enable]) | ||||
381 | |||||
382 | =item $enabled = $json->get_ascii | ||||
383 | |||||
384 | If C<$enable> is true (or missing), then the C<encode> method will not | ||||
385 | generate characters outside the code range C<0..127> (which is ASCII). Any | ||||
386 | Unicode characters outside that range will be escaped using either a | ||||
387 | single C<\uXXXX> (BMP characters) or a double C<\uHHHH\uLLLLL> escape sequence, | ||||
388 | as per RFC4627. The resulting encoded JSON text can be treated as a native | ||||
389 | Unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string, | ||||
390 | or any other superset of ASCII. | ||||
391 | |||||
392 | If C<$enable> is false, then the C<encode> method will not escape Unicode | ||||
393 | characters unless required by the JSON syntax or other flags. This results | ||||
394 | in a faster and more compact format. | ||||
395 | |||||
396 | See also the section I<ENCODING/CODESET FLAG NOTES> later in this | ||||
397 | document. | ||||
398 | |||||
399 | The main use for this flag is to produce JSON texts that can be | ||||
400 | transmitted over a 7-bit channel, as the encoded JSON texts will not | ||||
401 | contain any 8 bit characters. | ||||
402 | |||||
403 | Cpanel::JSON::XS->new->ascii (1)->encode ([chr 0x10401]) | ||||
404 | => ["\ud801\udc01"] | ||||
405 | |||||
406 | =item $json = $json->latin1 ([$enable]) | ||||
407 | |||||
408 | =item $enabled = $json->get_latin1 | ||||
409 | |||||
410 | If C<$enable> is true (or missing), then the C<encode> method will encode | ||||
411 | the resulting JSON text as latin1 (or ISO-8859-1), escaping any characters | ||||
412 | outside the code range C<0..255>. The resulting string can be treated as a | ||||
413 | latin1-encoded JSON text or a native Unicode string. The C<decode> method | ||||
414 | will not be affected in any way by this flag, as C<decode> by default | ||||
415 | expects Unicode, which is a strict superset of latin1. | ||||
416 | |||||
417 | If C<$enable> is false, then the C<encode> method will not escape Unicode | ||||
418 | characters unless required by the JSON syntax or other flags. | ||||
419 | |||||
420 | See also the section I<ENCODING/CODESET FLAG NOTES> later in this | ||||
421 | document. | ||||
422 | |||||
423 | The main use for this flag is efficiently encoding binary data as JSON | ||||
424 | text, as most octets will not be escaped, resulting in a smaller encoded | ||||
425 | size. The disadvantage is that the resulting JSON text is encoded | ||||
426 | in latin1 (and must correctly be treated as such when storing and | ||||
427 | transferring), a rare encoding for JSON. It is therefore most useful when | ||||
428 | you want to store data structures known to contain binary data efficiently | ||||
429 | in files or databases, not when talking to other JSON encoders/decoders. | ||||
430 | |||||
431 | Cpanel::JSON::XS->new->latin1->encode (["\x{89}\x{abc}"] | ||||
432 | => ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not) | ||||
433 | |||||
434 | |||||
435 | =item $json = $json->binary ([$enable]) | ||||
436 | |||||
437 | =item $enabled = $json = $json->get_binary | ||||
438 | |||||
439 | If the C<$enable> argument is true (or missing), then the C<encode> | ||||
440 | method will not try to detect an UTF-8 encoding in any JSON string, it | ||||
441 | will strictly interpret it as byte sequence. The result might contain | ||||
442 | new C<\xNN> sequences, which is B<unparsable JSON>. The C<decode> | ||||
443 | method forbids C<\uNNNN> sequences and accepts C<\xNN> and octal | ||||
444 | C<\NNN> sequences. | ||||
445 | |||||
446 | There is also a special logic for perl 5.6 and utf8. 5.6 encodes any | ||||
447 | string to utf-8 automatically when seeing a codepoint >= C<0x80> and | ||||
448 | < C<0x100>. With the binary flag enabled decode the perl utf8 encoded | ||||
449 | string to the original byte encoding and encode this with C<\xNN> | ||||
450 | escapes. This will result to the same encodings as with newer | ||||
451 | perls. But note that binary multi-byte codepoints with 5.6 will | ||||
452 | result in C<illegal unicode character in binary string> errors, | ||||
453 | unlike with newer perls. | ||||
454 | |||||
455 | If C<$enable> is false, then the C<encode> method will smartly try to | ||||
456 | detect Unicode characters unless required by the JSON syntax or other | ||||
457 | flags and hex and octal sequences are forbidden. | ||||
458 | |||||
459 | See also the section I<ENCODING/CODESET FLAG NOTES> later in this | ||||
460 | document. | ||||
461 | |||||
462 | The main use for this flag is to avoid the smart unicode detection and | ||||
463 | possible double encoding. The disadvantage is that the resulting JSON | ||||
464 | text is encoded in new C<\xNN> and in latin1 characters and must | ||||
465 | correctly be treated as such when storing and transferring, a rare | ||||
466 | encoding for JSON. It will produce non-readable JSON strings in the | ||||
467 | browser. It is therefore most useful when you want to store data | ||||
468 | structures known to contain binary data efficiently in files or | ||||
469 | databases, not when talking to other JSON encoders/decoders. The | ||||
470 | binary decoding method can also be used when an encoder produced a | ||||
471 | non-JSON conformant hex or octal encoding C<\xNN> or C<\NNN>. | ||||
472 | |||||
473 | Cpanel::JSON::XS->new->binary->encode (["\x{89}\x{abc}"]) | ||||
474 | 5.6: Error: malformed or illegal unicode character in binary string | ||||
475 | >=5.8: ['\x89\xe0\xaa\xbc'] | ||||
476 | |||||
477 | Cpanel::JSON::XS->new->binary->encode (["\x{89}\x{bc}"]) | ||||
478 | => ["\x89\xbc"] | ||||
479 | |||||
480 | Cpanel::JSON::XS->new->binary->decode (["\x89\ua001"]) | ||||
481 | Error: malformed or illegal unicode character in binary string | ||||
482 | |||||
483 | Cpanel::JSON::XS->new->decode (["\x89"]) | ||||
484 | Error: illegal hex character in non-binary string | ||||
485 | |||||
486 | |||||
487 | =item $json = $json->utf8 ([$enable]) | ||||
488 | |||||
489 | =item $enabled = $json->get_utf8 | ||||
490 | |||||
491 | If C<$enable> is true (or missing), then the C<encode> method will encode | ||||
492 | the JSON result into UTF-8, as required by many protocols, while the | ||||
493 | C<decode> method expects to be handled an UTF-8-encoded string. Please | ||||
494 | note that UTF-8-encoded strings do not contain any characters outside the | ||||
495 | range C<0..255>, they are thus useful for bytewise/binary I/O. In future | ||||
496 | versions, enabling this option might enable autodetection of the UTF-16 | ||||
497 | and UTF-32 encoding families, as described in RFC4627. | ||||
498 | |||||
499 | If C<$enable> is false, then the C<encode> method will return the JSON | ||||
500 | string as a (non-encoded) Unicode string, while C<decode> expects thus a | ||||
501 | Unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs | ||||
502 | to be done yourself, e.g. using the Encode module. | ||||
503 | |||||
504 | See also the section I<ENCODING/CODESET FLAG NOTES> later in this | ||||
505 | document. | ||||
506 | |||||
507 | Example, output UTF-16BE-encoded JSON: | ||||
508 | |||||
509 | use Encode; | ||||
510 | $jsontext = encode "UTF-16BE", Cpanel::JSON::XS->new->encode ($object); | ||||
511 | |||||
512 | Example, decode UTF-32LE-encoded JSON: | ||||
513 | |||||
514 | use Encode; | ||||
515 | $object = Cpanel::JSON::XS->new->decode (decode "UTF-32LE", $jsontext); | ||||
516 | |||||
517 | =item $json = $json->pretty ([$enable]) | ||||
518 | |||||
519 | This enables (or disables) all of the C<indent>, C<space_before> and | ||||
520 | C<space_after> (and in the future possibly more) flags in one call to | ||||
521 | generate the most readable (or most compact) form possible. | ||||
522 | |||||
523 | Example, pretty-print some simple structure: | ||||
524 | |||||
525 | my $json = Cpanel::JSON::XS->new->pretty(1)->encode ({a => [1,2]}) | ||||
526 | => | ||||
527 | { | ||||
528 | "a" : [ | ||||
529 | 1, | ||||
530 | 2 | ||||
531 | ] | ||||
532 | } | ||||
533 | |||||
534 | |||||
535 | =item $json = $json->indent ([$enable]) | ||||
536 | |||||
537 | =item $enabled = $json->get_indent | ||||
538 | |||||
539 | If C<$enable> is true (or missing), then the C<encode> method will use | ||||
540 | a multiline format as output, putting every array member or | ||||
541 | object/hash key-value pair into its own line, indenting them properly. | ||||
542 | |||||
543 | If C<$enable> is false, no newlines or indenting will be produced, and the | ||||
544 | resulting JSON text is guaranteed not to contain any C<newlines>. | ||||
545 | |||||
546 | This setting has no effect when decoding JSON texts. | ||||
547 | |||||
548 | =item $json = $json->indent_length([$number_of_spaces]) | ||||
549 | |||||
550 | =item $length = $json->get_indent_length() | ||||
551 | |||||
552 | Set the indent length (default C<3>). | ||||
553 | This option is only useful when you also enable indent or pretty. | ||||
554 | The acceptable range is from 0 (no indentation) to 15 | ||||
555 | |||||
556 | =item $json = $json->space_before ([$enable]) | ||||
557 | |||||
558 | =item $enabled = $json->get_space_before | ||||
559 | |||||
560 | If C<$enable> is true (or missing), then the C<encode> method will add an extra | ||||
561 | optional space before the C<:> separating keys from values in JSON objects. | ||||
562 | |||||
563 | If C<$enable> is false, then the C<encode> method will not add any extra | ||||
564 | space at those places. | ||||
565 | |||||
566 | This setting has no effect when decoding JSON texts. You will also | ||||
567 | most likely combine this setting with C<space_after>. | ||||
568 | |||||
569 | Example, space_before enabled, space_after and indent disabled: | ||||
570 | |||||
571 | {"key" :"value"} | ||||
572 | |||||
573 | =item $json = $json->space_after ([$enable]) | ||||
574 | |||||
575 | =item $enabled = $json->get_space_after | ||||
576 | |||||
577 | If C<$enable> is true (or missing), then the C<encode> method will add | ||||
578 | an extra optional space after the C<:> separating keys from values in | ||||
579 | JSON objects and extra whitespace after the C<,> separating key-value | ||||
580 | pairs and array members. | ||||
581 | |||||
582 | If C<$enable> is false, then the C<encode> method will not add any extra | ||||
583 | space at those places. | ||||
584 | |||||
585 | This setting has no effect when decoding JSON texts. | ||||
586 | |||||
587 | Example, space_before and indent disabled, space_after enabled: | ||||
588 | |||||
589 | {"key": "value"} | ||||
590 | |||||
591 | =item $json = $json->relaxed ([$enable]) | ||||
592 | |||||
593 | =item $enabled = $json->get_relaxed | ||||
594 | |||||
595 | If C<$enable> is true (or missing), then C<decode> will accept some | ||||
596 | extensions to normal JSON syntax (see below). C<encode> will not be | ||||
597 | affected in anyway. I<Be aware that this option makes you accept invalid | ||||
598 | JSON texts as if they were valid!>. I suggest only to use this option to | ||||
599 | parse application-specific files written by humans (configuration files, | ||||
600 | resource files etc.) | ||||
601 | |||||
602 | If C<$enable> is false (the default), then C<decode> will only accept | ||||
603 | valid JSON texts. | ||||
604 | |||||
605 | Currently accepted extensions are: | ||||
606 | |||||
607 | =over 4 | ||||
608 | |||||
609 | =item * list items can have an end-comma | ||||
610 | |||||
611 | JSON I<separates> array elements and key-value pairs with commas. This | ||||
612 | can be annoying if you write JSON texts manually and want to be able to | ||||
613 | quickly append elements, so this extension accepts comma at the end of | ||||
614 | such items not just between them: | ||||
615 | |||||
616 | [ | ||||
617 | 1, | ||||
618 | 2, <- this comma not normally allowed | ||||
619 | ] | ||||
620 | { | ||||
621 | "k1": "v1", | ||||
622 | "k2": "v2", <- this comma not normally allowed | ||||
623 | } | ||||
624 | |||||
625 | =item * shell-style '#'-comments | ||||
626 | |||||
627 | Whenever JSON allows whitespace, shell-style comments are additionally | ||||
628 | allowed. They are terminated by the first carriage-return or line-feed | ||||
629 | character, after which more white-space and comments are allowed. | ||||
630 | |||||
631 | [ | ||||
632 | 1, # this comment not allowed in JSON | ||||
633 | # neither this one... | ||||
634 | ] | ||||
635 | |||||
636 | =item * literal ASCII TAB characters in strings | ||||
637 | |||||
638 | Literal ASCII TAB characters are now allowed in strings (and treated as | ||||
639 | C<\t>) in relaxed mode. Despite JSON mandates, that TAB character is | ||||
640 | substituted for "\t" sequence. | ||||
641 | |||||
642 | [ | ||||
643 | "Hello\tWorld", | ||||
644 | "Hello<TAB>World", # literal <TAB> would not normally be allowed | ||||
645 | ] | ||||
646 | |||||
647 | =item * allow_singlequote | ||||
648 | |||||
649 | Single quotes are accepted instead of double quotes. See the | ||||
650 | L</allow_singlequote> option. | ||||
651 | |||||
652 | { "foo":'bar' } | ||||
653 | { 'foo':"bar" } | ||||
654 | { 'foo':'bar' } | ||||
655 | |||||
656 | =item * allow_barekey | ||||
657 | |||||
658 | Accept unquoted object keys instead of with mandatory double quotes. See the | ||||
659 | L</allow_barekey> option. | ||||
660 | |||||
661 | { foo:"bar" } | ||||
662 | |||||
663 | =item * allow_dupkeys | ||||
664 | |||||
665 | Allow decoding of duplicate keys in hashes. By default duplicate keys are forbidden. | ||||
666 | See L<http://seriot.ch/parsing_json.php#24>: | ||||
667 | RFC 7159 section 4: "The names within an object should be unique." | ||||
668 | See the C<allow_dupkeys> option. | ||||
669 | |||||
670 | =back | ||||
671 | |||||
672 | =item $json = $json->canonical ([$enable]) | ||||
673 | |||||
674 | =item $enabled = $json->get_canonical | ||||
675 | |||||
676 | If C<$enable> is true (or missing), then the C<encode> method will | ||||
677 | output JSON objects by sorting their keys. This is adding a | ||||
678 | comparatively high overhead. | ||||
679 | |||||
680 | If C<$enable> is false, then the C<encode> method will output key-value | ||||
681 | pairs in the order Perl stores them (which will likely change between runs | ||||
682 | of the same script, and can change even within the same run from 5.18 | ||||
683 | onwards). | ||||
684 | |||||
685 | This option is useful if you want the same data structure to be encoded as | ||||
686 | the same JSON text (given the same overall settings). If it is disabled, | ||||
687 | the same hash might be encoded differently even if contains the same data, | ||||
688 | as key-value pairs have no inherent ordering in Perl. | ||||
689 | |||||
690 | This setting has no effect when decoding JSON texts. | ||||
691 | |||||
692 | This is now also done with tied hashes, contrary to L<JSON::XS>. | ||||
693 | But note that with most large tied hashes stored as tree it is advised to | ||||
694 | sort the iterator already and don't sort the hash output here. Most such | ||||
695 | iterators are already sorted, as such e.g. L<DB_File> with C<DB_BTREE>. | ||||
696 | |||||
697 | =item $json = $json->sort_by (undef, 0, 1 or a block) | ||||
698 | |||||
699 | This currently only (un)sets the C<canonical> option, and ignores | ||||
700 | custom sort blocks. | ||||
701 | |||||
702 | This setting has no effect when decoding JSON texts. | ||||
703 | |||||
704 | This setting has currently no effect on tied hashes. | ||||
705 | |||||
706 | |||||
707 | =item $json = $json->escape_slash ([$enable]) | ||||
708 | |||||
709 | =item $enabled = $json->get_escape_slash | ||||
710 | |||||
711 | According to the JSON Grammar, the I<forward slash> character (U+002F) | ||||
712 | C<"/"> need to be escaped. But by default strings are encoded without | ||||
713 | escaping slashes in all perl JSON encoders. | ||||
714 | |||||
715 | If C<$enable> is true (or missing), then C<encode> will escape slashes, | ||||
716 | C<"\/">. | ||||
717 | |||||
718 | This setting has no effect when decoding JSON texts. | ||||
719 | |||||
720 | |||||
721 | =item $json = $json->unblessed_bool ([$enable]) | ||||
722 | |||||
723 | =item $enabled = $json->get_unblessed_bool | ||||
724 | |||||
725 | $json = $json->unblessed_bool([$enable]) | ||||
726 | |||||
727 | If C<$enable> is true (or missing), then C<decode> will return | ||||
728 | Perl non-object boolean variables (1 and 0) for JSON booleans | ||||
729 | (C<true> and C<false>). If C<$enable> is false, then C<decode> | ||||
730 | will return C<JSON::PP::Boolean> objects for JSON booleans. | ||||
731 | |||||
732 | |||||
733 | =item $json = $json->allow_singlequote ([$enable]) | ||||
734 | |||||
735 | =item $enabled = $json->get_allow_singlequote | ||||
736 | |||||
737 | $json = $json->allow_singlequote([$enable]) | ||||
738 | |||||
739 | If C<$enable> is true (or missing), then C<decode> will accept | ||||
740 | JSON strings quoted by single quotations that are invalid JSON | ||||
741 | format. | ||||
742 | |||||
743 | $json->allow_singlequote->decode({"foo":'bar'}); | ||||
744 | $json->allow_singlequote->decode({'foo':"bar"}); | ||||
745 | $json->allow_singlequote->decode({'foo':'bar'}); | ||||
746 | |||||
747 | This is also enabled with C<relaxed>. | ||||
748 | As same as the C<relaxed> option, this option may be used to parse | ||||
749 | application-specific files written by humans. | ||||
750 | |||||
751 | |||||
752 | =item $json = $json->allow_barekey ([$enable]) | ||||
753 | |||||
754 | =item $enabled = $json->get_allow_barekey | ||||
755 | |||||
756 | $json = $json->allow_barekey([$enable]) | ||||
757 | |||||
758 | If C<$enable> is true (or missing), then C<decode> will accept | ||||
759 | bare keys of JSON object that are invalid JSON format. | ||||
760 | |||||
761 | Same as with the C<relaxed> option, this option may be used to parse | ||||
762 | application-specific files written by humans. | ||||
763 | |||||
764 | $json->allow_barekey->decode('{foo:"bar"}'); | ||||
765 | |||||
766 | =item $json = $json->allow_bignum ([$enable]) | ||||
767 | |||||
768 | =item $enabled = $json->get_allow_bignum | ||||
769 | |||||
770 | $json = $json->allow_bignum([$enable]) | ||||
771 | |||||
772 | If C<$enable> is true (or missing), then C<decode> will convert | ||||
773 | the big integer Perl cannot handle as integer into a L<Math::BigInt> | ||||
774 | object and convert a floating number (any) into a L<Math::BigFloat>. | ||||
775 | |||||
776 | On the contrary, C<encode> converts C<Math::BigInt> objects and | ||||
777 | C<Math::BigFloat> objects into JSON numbers with C<allow_blessed> | ||||
778 | enable. | ||||
779 | |||||
780 | $json->allow_nonref->allow_blessed->allow_bignum; | ||||
781 | $bigfloat = $json->decode('2.000000000000000000000000001'); | ||||
782 | print $json->encode($bigfloat); | ||||
783 | # => 2.000000000000000000000000001 | ||||
784 | |||||
785 | See L</MAPPING> about the normal conversion of JSON number. | ||||
786 | |||||
787 | |||||
788 | =item $json = $json->allow_bigint ([$enable]) | ||||
789 | |||||
790 | This option is obsolete and replaced by allow_bignum. | ||||
791 | |||||
792 | |||||
793 | =item $json = $json->allow_nonref ([$enable]) | ||||
794 | |||||
795 | =item $enabled = $json->get_allow_nonref | ||||
796 | |||||
797 | If C<$enable> is true (or missing), then the C<encode> method can | ||||
798 | convert a non-reference into its corresponding string, number or null | ||||
799 | JSON value, which is an extension to RFC4627. Likewise, C<decode> will | ||||
800 | accept those JSON values instead of croaking. | ||||
801 | |||||
802 | If C<$enable> is false, then the C<encode> method will croak if it isn't | ||||
803 | passed an arrayref or hashref, as JSON texts must either be an object | ||||
804 | or array. Likewise, C<decode> will croak if given something that is not a | ||||
805 | JSON object or array. | ||||
806 | |||||
807 | Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, | ||||
808 | resulting in an invalid JSON text: | ||||
809 | |||||
810 | Cpanel::JSON::XS->new->allow_nonref->encode ("Hello, World!") | ||||
811 | => "Hello, World!" | ||||
812 | |||||
813 | =item $json = $json->allow_unknown ([$enable]) | ||||
814 | |||||
815 | =item $enabled = $json->get_allow_unknown | ||||
816 | |||||
817 | If C<$enable> is true (or missing), then C<encode> will I<not> throw an | ||||
818 | exception when it encounters values it cannot represent in JSON (for | ||||
819 | example, filehandles) but instead will encode a JSON C<null> value. Note | ||||
820 | that blessed objects are not included here and are handled separately by | ||||
821 | c<allow_nonref>. | ||||
822 | |||||
823 | If C<$enable> is false (the default), then C<encode> will throw an | ||||
824 | exception when it encounters anything it cannot encode as JSON. | ||||
825 | |||||
826 | This option does not affect C<decode> in any way, and it is recommended to | ||||
827 | leave it off unless you know your communications partner. | ||||
828 | |||||
829 | =item $json = $json->allow_stringify ([$enable]) | ||||
830 | |||||
831 | =item $enabled = $json->get_allow_stringify | ||||
832 | |||||
833 | If C<$enable> is true (or missing), then C<encode> will stringify the | ||||
834 | non-object perl value or reference. Note that blessed objects are not | ||||
835 | included here and are handled separately by C<allow_blessed> and | ||||
836 | C<convert_blessed>. String references are stringified to the string | ||||
837 | value, other references as in perl. | ||||
838 | |||||
839 | This option does not affect C<decode> in any way. | ||||
840 | |||||
841 | This option is special to this module, it is not supported by other | ||||
842 | encoders. So it is not recommended to use it. | ||||
843 | |||||
844 | =item $json = $json->require_types ([$enable]) | ||||
845 | |||||
846 | =item $enable = $json->get_require_types | ||||
847 | |||||
848 | $json = $json->require_types([$enable]) | ||||
849 | |||||
850 | If C<$enable> is true (or missing), then C<encode> will require | ||||
851 | either enabled C<type_all_string> or second argument with supplied JSON types. | ||||
852 | See L<Cpanel::JSON::XS::Type>. When C<type_all_string> is not enabled or | ||||
853 | second argument is not provided (or is undef), then C<encode> | ||||
854 | croaks. It also croaks when the type for provided structure in | ||||
855 | C<encode> is incomplete. | ||||
856 | |||||
857 | =item $json = $json->type_all_string ([$enable]) | ||||
858 | |||||
859 | =item $enable = $json->get_type_all_string | ||||
860 | |||||
861 | $json = $json->type_all_string([$enable]) | ||||
862 | |||||
863 | If C<$enable> is true (or missing), then C<encode> will always | ||||
864 | produce stable deterministic JSON string types in resulted output. | ||||
865 | |||||
866 | When C<$enable> is false, then result of encoded JSON output may be | ||||
867 | different for different Perl versions and may depends on loaded modules. | ||||
868 | |||||
869 | This is useful it you need deterministic JSON types, independently of used | ||||
870 | Perl version and other modules, but do not want to write complicated type | ||||
871 | definitions for L<Cpanel::JSON::XS::Type>. | ||||
872 | |||||
873 | =item $json = $json->allow_dupkeys ([$enable]) | ||||
874 | |||||
875 | =item $enabled = $json->get_allow_dupkeys | ||||
876 | |||||
877 | If C<$enable> is true (or missing), then the C<decode> method will not | ||||
878 | die when it encounters duplicate keys in a hash. | ||||
879 | C<allow_dupkeys> is also enabled in the C<relaxed> mode. | ||||
880 | |||||
881 | The JSON spec allows duplicate name in objects but recommends to | ||||
882 | disable it, however with Perl hashes they are impossible, parsing | ||||
883 | JSON in Perl silently ignores duplicate names, using the last value | ||||
884 | found. | ||||
885 | |||||
886 | See L<http://seriot.ch/parsing_json.php#24>: | ||||
887 | RFC 7159 section 4: "The names within an object should be unique." | ||||
888 | |||||
889 | =item $json = $json->dupkeys_as_arrayref ([$enable]) | ||||
890 | |||||
891 | =item $enabled = $json->get_dupkeys_as_arrayref | ||||
892 | |||||
893 | If enabled, allow decoding of duplicate keys in hashes and store the | ||||
894 | values as arrayref in the hash instead. By default duplicate keys are | ||||
895 | forbidden. Enabling this also enables the L</allow_dupkeys> option, | ||||
896 | but disabling this does not disable the L</allow_dupkeys> option. | ||||
897 | |||||
898 | Example: | ||||
899 | |||||
900 | $json->dupkeys_as_arrayref; | ||||
901 | print encode_json ($json->decode ('{"a":"b","a":"c"}')); | ||||
902 | |||||
903 | => {"a":["b","c"]} | ||||
904 | |||||
905 | This changes the result structure, thus cannot be enabled by default. | ||||
906 | The client must be aware of it. The resulting arrayref is not yet marked somehow | ||||
907 | (blessed or such). | ||||
908 | |||||
909 | =item $json = $json->allow_blessed ([$enable]) | ||||
910 | |||||
911 | =item $enabled = $json->get_allow_blessed | ||||
912 | |||||
913 | If C<$enable> is true (or missing), then the C<encode> method will not | ||||
914 | barf when it encounters a blessed reference. Instead, the value of the | ||||
915 | B<convert_blessed> option will decide whether C<null> (C<convert_blessed> | ||||
916 | disabled or no C<TO_JSON> method found) or a representation of the | ||||
917 | object (C<convert_blessed> enabled and C<TO_JSON> method found) is being | ||||
918 | encoded. Has no effect on C<decode>. | ||||
919 | |||||
920 | If C<$enable> is false (the default), then C<encode> will throw an | ||||
921 | exception when it encounters a blessed object without C<convert_blessed> | ||||
922 | and a C<TO_JSON> method. | ||||
923 | |||||
924 | This setting has no effect on C<decode>. | ||||
925 | |||||
926 | =item $json = $json->convert_blessed ([$enable]) | ||||
927 | |||||
928 | =item $enabled = $json->get_convert_blessed | ||||
929 | |||||
930 | If C<$enable> is true (or missing), then C<encode>, upon encountering a | ||||
931 | blessed object, will check for the availability of the C<TO_JSON> method | ||||
932 | on the object's class. If found, it will be called in scalar context | ||||
933 | and the resulting scalar will be encoded instead of the object. If no | ||||
934 | C<TO_JSON> method is found, a stringification overload method is tried next. | ||||
935 | If both are not found, the value of C<allow_blessed> will decide what | ||||
936 | to do. | ||||
937 | |||||
938 | The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> | ||||
939 | returns other blessed objects, those will be handled in the same | ||||
940 | way. C<TO_JSON> must take care of not causing an endless recursion | ||||
941 | cycle (== crash) in this case. The same care must be taken with | ||||
942 | calling encode in stringify overloads (even if this works by luck in | ||||
943 | older perls) or other callbacks. The name of C<TO_JSON> was chosen | ||||
944 | because other methods called by the Perl core (== not by the user of | ||||
945 | the object) are usually in upper case letters and to avoid collisions | ||||
946 | with any C<to_json> function or method. | ||||
947 | |||||
948 | If C<$enable> is false (the default), then C<encode> will not consider | ||||
949 | this type of conversion. | ||||
950 | |||||
951 | This setting has no effect on C<decode>. | ||||
952 | |||||
953 | =item $json = $json->allow_tags ([$enable]) | ||||
954 | |||||
955 | =item $enabled = $json->get_allow_tags | ||||
956 | |||||
957 | See L<OBJECT SERIALIZATION> for details. | ||||
958 | |||||
959 | If C<$enable> is true (or missing), then C<encode>, upon encountering a | ||||
960 | blessed object, will check for the availability of the C<FREEZE> method on | ||||
961 | the object's class. If found, it will be used to serialize the object into | ||||
962 | a nonstandard tagged JSON value (that JSON decoders cannot decode). | ||||
963 | |||||
964 | It also causes C<decode> to parse such tagged JSON values and deserialize | ||||
965 | them via a call to the C<THAW> method. | ||||
966 | |||||
967 | If C<$enable> is false (the default), then C<encode> will not consider | ||||
968 | this type of conversion, and tagged JSON values will cause a parse error | ||||
969 | in C<decode>, as if tags were not part of the grammar. | ||||
970 | |||||
971 | =item $json = $json->filter_json_object ([$coderef->($hashref)]) | ||||
972 | |||||
973 | When C<$coderef> is specified, it will be called from C<decode> each | ||||
974 | time it decodes a JSON object. The only argument is a reference to the | ||||
975 | newly-created hash. If the code references returns a single scalar (which | ||||
976 | need not be a reference), this value (i.e. a copy of that scalar to avoid | ||||
977 | aliasing) is inserted into the deserialized data structure. If it returns | ||||
978 | an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the | ||||
979 | original deserialized hash will be inserted. This setting can slow down | ||||
980 | decoding considerably. | ||||
981 | |||||
982 | When C<$coderef> is omitted or undefined, any existing callback will | ||||
983 | be removed and C<decode> will not change the deserialized hash in any | ||||
984 | way. | ||||
985 | |||||
986 | Example, convert all JSON objects into the integer 5: | ||||
987 | |||||
988 | my $js = Cpanel::JSON::XS->new->filter_json_object (sub { 5 }); | ||||
989 | # returns [5] | ||||
990 | $js->decode ('[{}]') | ||||
991 | # throw an exception because allow_nonref is not enabled | ||||
992 | # so a lone 5 is not allowed. | ||||
993 | $js->decode ('{"a":1, "b":2}'); | ||||
994 | |||||
995 | =item $json = $json->filter_json_single_key_object ($key [=> $coderef->($value)]) | ||||
996 | |||||
997 | Works remotely similar to C<filter_json_object>, but is only called for | ||||
998 | JSON objects having a single key named C<$key>. | ||||
999 | |||||
1000 | This C<$coderef> is called before the one specified via | ||||
1001 | C<filter_json_object>, if any. It gets passed the single value in the JSON | ||||
1002 | object. If it returns a single value, it will be inserted into the data | ||||
1003 | structure. If it returns nothing (not even C<undef> but the empty list), | ||||
1004 | the callback from C<filter_json_object> will be called next, as if no | ||||
1005 | single-key callback were specified. | ||||
1006 | |||||
1007 | If C<$coderef> is omitted or undefined, the corresponding callback will be | ||||
1008 | disabled. There can only ever be one callback for a given key. | ||||
1009 | |||||
1010 | As this callback gets called less often then the C<filter_json_object> | ||||
1011 | one, decoding speed will not usually suffer as much. Therefore, single-key | ||||
1012 | objects make excellent targets to serialize Perl objects into, especially | ||||
1013 | as single-key JSON objects are as close to the type-tagged value concept | ||||
1014 | as JSON gets (it's basically an ID/VALUE tuple). Of course, JSON does not | ||||
1015 | support this in any way, so you need to make sure your data never looks | ||||
1016 | like a serialized Perl hash. | ||||
1017 | |||||
1018 | Typical names for the single object key are C<__class_whatever__>, or | ||||
1019 | C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even | ||||
1020 | things like C<__class_md5sum(classname)__>, to reduce the risk of clashing | ||||
1021 | with real hashes. | ||||
1022 | |||||
1023 | Example, decode JSON objects of the form C<< { "__widget__" => <id> } >> | ||||
1024 | into the corresponding C<< $WIDGET{<id>} >> object: | ||||
1025 | |||||
1026 | # return whatever is in $WIDGET{5}: | ||||
1027 | Cpanel::JSON::XS | ||||
1028 | ->new | ||||
1029 | ->filter_json_single_key_object (__widget__ => sub { | ||||
1030 | $WIDGET{ $_[0] } | ||||
1031 | }) | ||||
1032 | ->decode ('{"__widget__": 5') | ||||
1033 | |||||
1034 | # this can be used with a TO_JSON method in some "widget" class | ||||
1035 | # for serialization to json: | ||||
1036 | sub WidgetBase::TO_JSON { | ||||
1037 | my ($self) = @_; | ||||
1038 | |||||
1039 | unless ($self->{id}) { | ||||
1040 | $self->{id} = ..get..some..id..; | ||||
1041 | $WIDGET{$self->{id}} = $self; | ||||
1042 | } | ||||
1043 | |||||
1044 | { __widget__ => $self->{id} } | ||||
1045 | } | ||||
1046 | |||||
1047 | =item $json = $json->shrink ([$enable]) | ||||
1048 | |||||
1049 | =item $enabled = $json->get_shrink | ||||
1050 | |||||
1051 | Perl usually over-allocates memory a bit when allocating space for | ||||
1052 | strings. This flag optionally resizes strings generated by either | ||||
1053 | C<encode> or C<decode> to their minimum size possible. This can save | ||||
1054 | memory when your JSON texts are either very very long or you have many | ||||
1055 | short strings. It will also try to downgrade any strings to octet-form | ||||
1056 | if possible: perl stores strings internally either in an encoding called | ||||
1057 | UTF-X or in octet-form. The latter cannot store everything but uses less | ||||
1058 | space in general (and some buggy Perl or C code might even rely on that | ||||
1059 | internal representation being used). | ||||
1060 | |||||
1061 | The actual definition of what shrink does might change in future versions, | ||||
1062 | but it will always try to save space at the expense of time. | ||||
1063 | |||||
1064 | If C<$enable> is true (or missing), the string returned by C<encode> will | ||||
1065 | be shrunk-to-fit, while all strings generated by C<decode> will also be | ||||
1066 | shrunk-to-fit. | ||||
1067 | |||||
1068 | If C<$enable> is false, then the normal perl allocation algorithms are used. | ||||
1069 | If you work with your data, then this is likely to be faster. | ||||
1070 | |||||
1071 | In the future, this setting might control other things, such as converting | ||||
1072 | strings that look like integers or floats into integers or floats | ||||
1073 | internally (there is no difference on the Perl level), saving space. | ||||
1074 | |||||
1075 | =item $json = $json->max_depth ([$maximum_nesting_depth]) | ||||
1076 | |||||
1077 | =item $max_depth = $json->get_max_depth | ||||
1078 | |||||
1079 | Sets the maximum nesting level (default C<512>) accepted while encoding | ||||
1080 | or decoding. If a higher nesting level is detected in JSON text or a Perl | ||||
1081 | data structure, then the encoder and decoder will stop and croak at that | ||||
1082 | point. | ||||
1083 | |||||
1084 | Nesting level is defined by number of hash- or arrayrefs that the encoder | ||||
1085 | needs to traverse to reach a given point or the number of C<{> or C<[> | ||||
1086 | characters without their matching closing parenthesis crossed to reach a | ||||
1087 | given character in a string. | ||||
1088 | |||||
1089 | Setting the maximum depth to one disallows any nesting, so that ensures | ||||
1090 | that the object is only a single hash/object or array. | ||||
1091 | |||||
1092 | If no argument is given, the highest possible setting will be used, which | ||||
1093 | is rarely useful. | ||||
1094 | |||||
1095 | Note that nesting is implemented by recursion in C. The default value has | ||||
1096 | been chosen to be as large as typical operating systems allow without | ||||
1097 | crashing. | ||||
1098 | |||||
1099 | See L</SECURITY CONSIDERATIONS>, below, for more info on why this is useful. | ||||
1100 | |||||
1101 | =item $json = $json->max_size ([$maximum_string_size]) | ||||
1102 | |||||
1103 | =item $max_size = $json->get_max_size | ||||
1104 | |||||
1105 | Set the maximum length a JSON text may have (in bytes) where decoding is | ||||
1106 | being attempted. The default is C<0>, meaning no limit. When C<decode> | ||||
1107 | is called on a string that is longer then this many bytes, it will not | ||||
1108 | attempt to decode the string but throw an exception. This setting has no | ||||
1109 | effect on C<encode> (yet). | ||||
1110 | |||||
1111 | If no argument is given, the limit check will be deactivated (same as when | ||||
1112 | C<0> is specified). | ||||
1113 | |||||
1114 | See L</SECURITY CONSIDERATIONS>, below, for more info on why this is useful. | ||||
1115 | |||||
1116 | =item $json->stringify_infnan ([$infnan_mode = 1]) | ||||
1117 | |||||
1118 | =item $infnan_mode = $json->get_stringify_infnan | ||||
1119 | |||||
1120 | Get or set how Cpanel::JSON::XS encodes C<inf>, C<-inf> or C<nan> for numeric | ||||
1121 | values. Also qnan, snan or negative nan on some platforms. | ||||
1122 | |||||
1123 | C<null>: infnan_mode = 0. Similar to most JSON modules in other languages. | ||||
1124 | Always null. | ||||
1125 | |||||
1126 | stringified: infnan_mode = 1. As in Mojo::JSON. Platform specific strings. | ||||
1127 | Stringified via sprintf(%g), with double quotes. | ||||
1128 | |||||
1129 | inf/nan: infnan_mode = 2. As in JSON::XS, and older releases. | ||||
1130 | Passes through platform dependent values, invalid JSON. Stringified via | ||||
1131 | sprintf(%g), but without double quotes. | ||||
1132 | |||||
1133 | "inf/-inf/nan": infnan_mode = 3. Platform independent inf/nan/-inf | ||||
1134 | strings. No QNAN/SNAN/negative NAN support, unified to "nan". Much | ||||
1135 | easier to detect, but may conflict with valid strings. | ||||
1136 | |||||
1137 | =item $json_text = $json->encode ($perl_scalar, $json_type) | ||||
1138 | |||||
1139 | Converts the given Perl data structure (a simple scalar or a reference | ||||
1140 | to a hash or array) to its JSON representation. Simple scalars will be | ||||
1141 | converted into JSON string or number sequences, while references to | ||||
1142 | arrays become JSON arrays and references to hashes become JSON | ||||
1143 | objects. Undefined Perl values (e.g. C<undef>) become JSON C<null> | ||||
1144 | values. Neither C<true> nor C<false> values will be generated. | ||||
1145 | |||||
1146 | For the type argument see L<Cpanel::JSON::XS::Type>. | ||||
1147 | |||||
1148 | =item $perl_scalar = $json->decode ($json_text, my $json_type) | ||||
1149 | |||||
1150 | The opposite of C<encode>: expects a JSON text and tries to parse it, | ||||
1151 | returning the resulting simple scalar or reference. Croaks on error. | ||||
1152 | |||||
1153 | JSON numbers and strings become simple Perl scalars. JSON arrays become | ||||
1154 | Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes | ||||
1155 | C<1>, C<false> becomes C<0> and C<null> becomes C<undef>. | ||||
1156 | |||||
1157 | For the type argument see L<Cpanel::JSON::XS::Type>. | ||||
1158 | |||||
1159 | =item ($perl_scalar, $characters) = $json->decode_prefix ($json_text) | ||||
1160 | |||||
1161 | This works like the C<decode> method, but instead of raising an exception | ||||
1162 | when there is trailing garbage after the first JSON object, it will | ||||
1163 | silently stop parsing there and return the number of characters consumed | ||||
1164 | so far. | ||||
1165 | |||||
1166 | This is useful if your JSON texts are not delimited by an outer protocol | ||||
1167 | and you need to know where the JSON text ends. | ||||
1168 | |||||
1169 | Cpanel::JSON::XS->new->decode_prefix ("[1] the tail") | ||||
1170 | => ([1], 3) | ||||
1171 | |||||
1172 | =item $json->to_json ($perl_hash_or_arrayref) | ||||
1173 | |||||
1174 | Deprecated method for perl 5.8 and newer. Use L<encode_json> instead. | ||||
1175 | |||||
1176 | =item $json->from_json ($utf8_encoded_json_text) | ||||
1177 | |||||
1178 | Deprecated method for perl 5.8 and newer. Use L<decode_json> instead. | ||||
1179 | |||||
1180 | =back | ||||
1181 | |||||
1182 | |||||
1183 | =head1 INCREMENTAL PARSING | ||||
1184 | |||||
1185 | In some cases, there is the need for incremental parsing of JSON | ||||
1186 | texts. While this module always has to keep both JSON text and resulting | ||||
1187 | Perl data structure in memory at one time, it does allow you to parse a | ||||
1188 | JSON stream incrementally. It does so by accumulating text until it has | ||||
1189 | a full JSON object, which it then can decode. This process is similar to | ||||
1190 | using C<decode_prefix> to see if a full JSON object is available, but | ||||
1191 | is much more efficient (and can be implemented with a minimum of method | ||||
1192 | calls). | ||||
1193 | |||||
1194 | Cpanel::JSON::XS will only attempt to parse the JSON text once it is | ||||
1195 | sure it has enough text to get a decisive result, using a very simple | ||||
1196 | but truly incremental parser. This means that it sometimes won't stop | ||||
1197 | as early as the full parser, for example, it doesn't detect mismatched | ||||
1198 | parentheses. The only thing it guarantees is that it starts decoding | ||||
1199 | as soon as a syntactically valid JSON text has been seen. This means | ||||
1200 | you need to set resource limits (e.g. C<max_size>) to ensure the | ||||
1201 | parser will stop parsing in the presence if syntax errors. | ||||
1202 | |||||
1203 | The following methods implement this incremental parser. | ||||
1204 | |||||
1205 | =over 4 | ||||
1206 | |||||
1207 | =item [void, scalar or list context] = $json->incr_parse ([$string]) | ||||
1208 | |||||
1209 | This is the central parsing function. It can both append new text and | ||||
1210 | extract objects from the stream accumulated so far (both of these | ||||
1211 | functions are optional). | ||||
1212 | |||||
1213 | If C<$string> is given, then this string is appended to the already | ||||
1214 | existing JSON fragment stored in the C<$json> object. | ||||
1215 | |||||
1216 | After that, if the function is called in void context, it will simply | ||||
1217 | return without doing anything further. This can be used to add more text | ||||
1218 | in as many chunks as you want. | ||||
1219 | |||||
1220 | If the method is called in scalar context, then it will try to extract | ||||
1221 | exactly I<one> JSON object. If that is successful, it will return this | ||||
1222 | object, otherwise it will return C<undef>. If there is a parse error, | ||||
1223 | this method will croak just as C<decode> would do (one can then use | ||||
1224 | C<incr_skip> to skip the erroneous part). This is the most common way of | ||||
1225 | using the method. | ||||
1226 | |||||
1227 | And finally, in list context, it will try to extract as many objects | ||||
1228 | from the stream as it can find and return them, or the empty list | ||||
1229 | otherwise. For this to work, there must be no separators between the JSON | ||||
1230 | objects or arrays, instead they must be concatenated back-to-back. If | ||||
1231 | an error occurs, an exception will be raised as in the scalar context | ||||
1232 | case. Note that in this case, any previously-parsed JSON texts will be | ||||
1233 | lost. | ||||
1234 | |||||
1235 | Example: Parse some JSON arrays/objects in a given string and return | ||||
1236 | them. | ||||
1237 | |||||
1238 | my @objs = Cpanel::JSON::XS->new->incr_parse ("[5][7][1,2]"); | ||||
1239 | |||||
1240 | =item $lvalue_string = $json->incr_text (>5.8 only) | ||||
1241 | |||||
1242 | This method returns the currently stored JSON fragment as an lvalue, that | ||||
1243 | is, you can manipulate it. This I<only> works when a preceding call to | ||||
1244 | C<incr_parse> in I<scalar context> successfully returned an object, and | ||||
1245 | 2. only with Perl >= 5.8 | ||||
1246 | |||||
1247 | Under all other circumstances you must not call this function (I mean | ||||
1248 | it. although in simple tests it might actually work, it I<will> fail | ||||
1249 | under real world conditions). As a special exception, you can also | ||||
1250 | call this method before having parsed anything. | ||||
1251 | |||||
1252 | This function is useful in two cases: a) finding the trailing text after a | ||||
1253 | JSON object or b) parsing multiple JSON objects separated by non-JSON text | ||||
1254 | (such as commas). | ||||
1255 | |||||
1256 | =item $json->incr_skip | ||||
1257 | |||||
1258 | This will reset the state of the incremental parser and will remove | ||||
1259 | the parsed text from the input buffer so far. This is useful after | ||||
1260 | C<incr_parse> died, in which case the input buffer and incremental parser | ||||
1261 | state is left unchanged, to skip the text parsed so far and to reset the | ||||
1262 | parse state. | ||||
1263 | |||||
1264 | The difference to C<incr_reset> is that only text until the parse error | ||||
1265 | occurred is removed. | ||||
1266 | |||||
1267 | =item $json->incr_reset | ||||
1268 | |||||
1269 | This completely resets the incremental parser, that is, after this call, | ||||
1270 | it will be as if the parser had never parsed anything. | ||||
1271 | |||||
1272 | This is useful if you want to repeatedly parse JSON objects and want to | ||||
1273 | ignore any trailing data, which means you have to reset the parser after | ||||
1274 | each successful decode. | ||||
1275 | |||||
1276 | =back | ||||
1277 | |||||
1278 | =head2 LIMITATIONS | ||||
1279 | |||||
1280 | All options that affect decoding are supported, except | ||||
1281 | C<allow_nonref>. The reason for this is that it cannot be made to work | ||||
1282 | sensibly: JSON objects and arrays are self-delimited, i.e. you can | ||||
1283 | concatenate them back to back and still decode them perfectly. This | ||||
1284 | does not hold true for JSON numbers, however. | ||||
1285 | |||||
1286 | For example, is the string C<1> a single JSON number, or is it simply | ||||
1287 | the start of C<12>? Or is C<12> a single JSON number, or the | ||||
1288 | concatenation of C<1> and C<2>? In neither case you can tell, and this | ||||
1289 | is why Cpanel::JSON::XS takes the conservative route and disallows | ||||
1290 | this case. | ||||
1291 | |||||
1292 | =head2 EXAMPLES | ||||
1293 | |||||
1294 | Some examples will make all this clearer. First, a simple example that | ||||
1295 | works similarly to C<decode_prefix>: We want to decode the JSON object at | ||||
1296 | the start of a string and identify the portion after the JSON object: | ||||
1297 | |||||
1298 | my $text = "[1,2,3] hello"; | ||||
1299 | |||||
1300 | my $json = new Cpanel::JSON::XS; | ||||
1301 | |||||
1302 | my $obj = $json->incr_parse ($text) | ||||
1303 | or die "expected JSON object or array at beginning of string"; | ||||
1304 | |||||
1305 | my $tail = $json->incr_text; | ||||
1306 | # $tail now contains " hello" | ||||
1307 | |||||
1308 | Easy, isn't it? | ||||
1309 | |||||
1310 | Now for a more complicated example: Imagine a hypothetical protocol where | ||||
1311 | you read some requests from a TCP stream, and each request is a JSON | ||||
1312 | array, without any separation between them (in fact, it is often useful to | ||||
1313 | use newlines as "separators", as these get interpreted as whitespace at | ||||
1314 | the start of the JSON text, which makes it possible to test said protocol | ||||
1315 | with C<telnet>...). | ||||
1316 | |||||
1317 | Here is how you'd do it (it is trivial to write this in an event-based | ||||
1318 | manner): | ||||
1319 | |||||
1320 | my $json = new Cpanel::JSON::XS; | ||||
1321 | |||||
1322 | # read some data from the socket | ||||
1323 | while (sysread $socket, my $buf, 4096) { | ||||
1324 | |||||
1325 | # split and decode as many requests as possible | ||||
1326 | for my $request ($json->incr_parse ($buf)) { | ||||
1327 | # act on the $request | ||||
1328 | } | ||||
1329 | } | ||||
1330 | |||||
1331 | Another complicated example: Assume you have a string with JSON objects | ||||
1332 | or arrays, all separated by (optional) comma characters (e.g. C<[1],[2], | ||||
1333 | [3]>). To parse them, we have to skip the commas between the JSON texts, | ||||
1334 | and here is where the lvalue-ness of C<incr_text> comes in useful: | ||||
1335 | |||||
1336 | my $text = "[1],[2], [3]"; | ||||
1337 | my $json = new Cpanel::JSON::XS; | ||||
1338 | |||||
1339 | # void context, so no parsing done | ||||
1340 | $json->incr_parse ($text); | ||||
1341 | |||||
1342 | # now extract as many objects as possible. note the | ||||
1343 | # use of scalar context so incr_text can be called. | ||||
1344 | while (my $obj = $json->incr_parse) { | ||||
1345 | # do something with $obj | ||||
1346 | |||||
1347 | # now skip the optional comma | ||||
1348 | $json->incr_text =~ s/^ \s* , //x; | ||||
1349 | } | ||||
1350 | |||||
1351 | Now lets go for a very complex example: Assume that you have a gigantic | ||||
1352 | JSON array-of-objects, many gigabytes in size, and you want to parse it, | ||||
1353 | but you cannot load it into memory fully (this has actually happened in | ||||
1354 | the real world :). | ||||
1355 | |||||
1356 | Well, you lost, you have to implement your own JSON parser. But | ||||
1357 | Cpanel::JSON::XS can still help you: You implement a (very simple) | ||||
1358 | array parser and let JSON decode the array elements, which are all | ||||
1359 | full JSON objects on their own (this wouldn't work if the array | ||||
1360 | elements could be JSON numbers, for example): | ||||
1361 | |||||
1362 | my $json = new Cpanel::JSON::XS; | ||||
1363 | |||||
1364 | # open the monster | ||||
1365 | open my $fh, "<bigfile.json" | ||||
1366 | or die "bigfile: $!"; | ||||
1367 | |||||
1368 | # first parse the initial "[" | ||||
1369 | for (;;) { | ||||
1370 | sysread $fh, my $buf, 65536 | ||||
1371 | or die "read error: $!"; | ||||
1372 | $json->incr_parse ($buf); # void context, so no parsing | ||||
1373 | |||||
1374 | # Exit the loop once we found and removed(!) the initial "[". | ||||
1375 | # In essence, we are (ab-)using the $json object as a simple scalar | ||||
1376 | # we append data to. | ||||
1377 | last if $json->incr_text =~ s/^ \s* \[ //x; | ||||
1378 | } | ||||
1379 | |||||
1380 | # now we have the skipped the initial "[", so continue | ||||
1381 | # parsing all the elements. | ||||
1382 | for (;;) { | ||||
1383 | # in this loop we read data until we got a single JSON object | ||||
1384 | for (;;) { | ||||
1385 | if (my $obj = $json->incr_parse) { | ||||
1386 | # do something with $obj | ||||
1387 | last; | ||||
1388 | } | ||||
1389 | |||||
1390 | # add more data | ||||
1391 | sysread $fh, my $buf, 65536 | ||||
1392 | or die "read error: $!"; | ||||
1393 | $json->incr_parse ($buf); # void context, so no parsing | ||||
1394 | } | ||||
1395 | |||||
1396 | # in this loop we read data until we either found and parsed the | ||||
1397 | # separating "," between elements, or the final "]" | ||||
1398 | for (;;) { | ||||
1399 | # first skip whitespace | ||||
1400 | $json->incr_text =~ s/^\s*//; | ||||
1401 | |||||
1402 | # if we find "]", we are done | ||||
1403 | if ($json->incr_text =~ s/^\]//) { | ||||
1404 | print "finished.\n"; | ||||
1405 | exit; | ||||
1406 | } | ||||
1407 | |||||
1408 | # if we find ",", we can continue with the next element | ||||
1409 | if ($json->incr_text =~ s/^,//) { | ||||
1410 | last; | ||||
1411 | } | ||||
1412 | |||||
1413 | # if we find anything else, we have a parse error! | ||||
1414 | if (length $json->incr_text) { | ||||
1415 | die "parse error near ", $json->incr_text; | ||||
1416 | } | ||||
1417 | |||||
1418 | # else add more data | ||||
1419 | sysread $fh, my $buf, 65536 | ||||
1420 | or die "read error: $!"; | ||||
1421 | $json->incr_parse ($buf); # void context, so no parsing | ||||
1422 | } | ||||
1423 | |||||
1424 | This is a complex example, but most of the complexity comes from the fact | ||||
1425 | that we are trying to be correct (bear with me if I am wrong, I never ran | ||||
1426 | the above example :). | ||||
1427 | |||||
1428 | =head1 BOM | ||||
1429 | |||||
1430 | Detect all unicode B<Byte Order Marks> on decode. | ||||
1431 | Which are UTF-8, UTF-16LE, UTF-16BE, UTF-32LE and UTF-32BE. | ||||
1432 | |||||
1433 | The BOM encoding is set only for one specific decode call, it does not | ||||
1434 | change the state of the JSON object. | ||||
1435 | |||||
1436 | B<Warning>: With perls older than 5.20 you need load the Encode module | ||||
1437 | before loading a multibyte BOM, i.e. >= UTF-16. Otherwise an error is | ||||
1438 | thrown. This is an implementation limitation and might get fixed later. | ||||
1439 | |||||
1440 | See L<https://tools.ietf.org/html/rfc7159#section-8.1> | ||||
1441 | I<"JSON text SHALL be encoded in UTF-8, UTF-16, or UTF-32."> | ||||
1442 | |||||
1443 | I<"Implementations MUST NOT add a byte order mark to the beginning of a | ||||
1444 | JSON text", "implementations (...) MAY ignore the presence of a byte | ||||
1445 | order mark rather than treating it as an error".> | ||||
1446 | |||||
1447 | See also L<http://www.unicode.org/faq/utf_bom.html#BOM>. | ||||
1448 | |||||
1449 | Beware that Cpanel::JSON::XS is currently the only JSON module which | ||||
1450 | does accept and decode a BOM. | ||||
1451 | |||||
1452 | The latest JSON spec | ||||
1453 | L<https://www.greenbytes.de/tech/webdav/rfc8259.html#character.encoding> | ||||
1454 | forbid the usage of UTF-16 or UTF-32, the character encoding is UTF-8. | ||||
1455 | Thus in subsequent updates BOM's of UTF-16 or UTF-32 will throw an error. | ||||
1456 | |||||
1457 | =head1 MAPPING | ||||
1458 | |||||
1459 | This section describes how Cpanel::JSON::XS maps Perl values to JSON | ||||
1460 | values and vice versa. These mappings are designed to "do the right | ||||
1461 | thing" in most circumstances automatically, preserving round-tripping | ||||
1462 | characteristics (what you put in comes out as something equivalent). | ||||
1463 | |||||
1464 | For the more enlightened: note that in the following descriptions, | ||||
1465 | lowercase I<perl> refers to the Perl interpreter, while uppercase I<Perl> | ||||
1466 | refers to the abstract Perl language itself. | ||||
1467 | |||||
1468 | |||||
1469 | =head2 JSON -> PERL | ||||
1470 | |||||
1471 | =over 4 | ||||
1472 | |||||
1473 | =item object | ||||
1474 | |||||
1475 | A JSON object becomes a reference to a hash in Perl. No ordering of object | ||||
1476 | keys is preserved (JSON does not preserve object key ordering itself). | ||||
1477 | |||||
1478 | =item array | ||||
1479 | |||||
1480 | A JSON array becomes a reference to an array in Perl. | ||||
1481 | |||||
1482 | =item string | ||||
1483 | |||||
1484 | A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON | ||||
1485 | are represented by the same codepoints in the Perl string, so no manual | ||||
1486 | decoding is necessary. | ||||
1487 | |||||
1488 | =item number | ||||
1489 | |||||
1490 | A JSON number becomes either an integer, numeric (floating point) or | ||||
1491 | string scalar in perl, depending on its range and any fractional parts. On | ||||
1492 | the Perl level, there is no difference between those as Perl handles all | ||||
1493 | the conversion details, but an integer may take slightly less memory and | ||||
1494 | might represent more values exactly than floating point numbers. | ||||
1495 | |||||
1496 | If the number consists of digits only, Cpanel::JSON::XS will try to | ||||
1497 | represent it as an integer value. If that fails, it will try to | ||||
1498 | represent it as a numeric (floating point) value if that is possible | ||||
1499 | without loss of precision. Otherwise it will preserve the number as a | ||||
1500 | string value (in which case you lose roundtripping ability, as the | ||||
1501 | JSON number will be re-encoded to a JSON string). | ||||
1502 | |||||
1503 | Numbers containing a fractional or exponential part will always be | ||||
1504 | represented as numeric (floating point) values, possibly at a loss of | ||||
1505 | precision (in which case you might lose perfect roundtripping ability, but | ||||
1506 | the JSON number will still be re-encoded as a JSON number). | ||||
1507 | |||||
1508 | Note that precision is not accuracy - binary floating point values | ||||
1509 | cannot represent most decimal fractions exactly, and when converting | ||||
1510 | from and to floating point, C<Cpanel::JSON::XS> only guarantees precision | ||||
1511 | up to but not including the least significant bit. | ||||
1512 | |||||
1513 | =item true, false | ||||
1514 | |||||
1515 | When C<unblessed_bool> is set to true, then JSON C<true> becomes C<1> and | ||||
1516 | JSON C<false> becomes C<0>. | ||||
1517 | |||||
1518 | Otherwise these JSON atoms become C<JSON::PP::true> and | ||||
1519 | C<JSON::PP::false>, respectively. They are C<JSON::PP::Boolean> | ||||
1520 | objects and are overloaded to act almost exactly like the numbers C<1> | ||||
1521 | and C<0>. You can check whether a scalar is a JSON boolean by using | ||||
1522 | the C<Cpanel::JSON::XS::is_bool> function. | ||||
1523 | |||||
1524 | The other round, from perl to JSON, C<!0> which is represented as | ||||
1525 | C<yes> becomes C<true>, and C<!1> which is represented as | ||||
1526 | C<no> becomes C<false>. | ||||
1527 | |||||
1528 | Via L<Cpanel::JSON::XS::Type> you can now even force negation in C<encode>, | ||||
1529 | without overloading of C<!>: | ||||
1530 | |||||
1531 | my $false = Cpanel::JSON::XS::false; | ||||
1532 | print($json->encode([!$false], [JSON_TYPE_BOOL])); | ||||
1533 | => [true] | ||||
1534 | |||||
1535 | =item null | ||||
1536 | |||||
1537 | A JSON null atom becomes C<undef> in Perl. | ||||
1538 | |||||
1539 | =item shell-style comments (C<< # I<text> >>) | ||||
1540 | |||||
1541 | As a nonstandard extension to the JSON syntax that is enabled by the | ||||
1542 | C<relaxed> setting, shell-style comments are allowed. They can start | ||||
1543 | anywhere outside strings and go till the end of the line. | ||||
1544 | |||||
1545 | =item tagged values (C<< (I<tag>)I<value> >>). | ||||
1546 | |||||
1547 | Another nonstandard extension to the JSON syntax, enabled with the | ||||
1548 | C<allow_tags> setting, are tagged values. In this implementation, the | ||||
1549 | I<tag> must be a perl package/class name encoded as a JSON string, and the | ||||
1550 | I<value> must be a JSON array encoding optional constructor arguments. | ||||
1551 | |||||
1552 | See L<OBJECT SERIALIZATION>, below, for details. | ||||
1553 | |||||
1554 | =back | ||||
1555 | |||||
1556 | |||||
1557 | =head2 PERL -> JSON | ||||
1558 | |||||
1559 | The mapping from Perl to JSON is slightly more difficult, as Perl is a | ||||
1560 | truly typeless language, so we can only guess which JSON type is meant by | ||||
1561 | a Perl value. | ||||
1562 | |||||
1563 | =over 4 | ||||
1564 | |||||
1565 | =item hash references | ||||
1566 | |||||
1567 | Perl hash references become JSON objects. As there is no inherent ordering | ||||
1568 | in hash keys (or JSON objects), they will usually be encoded in a | ||||
1569 | pseudo-random order that can change between runs of the same program but | ||||
1570 | stays generally the same within a single run of a program. Cpanel::JSON::XS can | ||||
1571 | optionally sort the hash keys (determined by the I<canonical> flag), so | ||||
1572 | the same datastructure will serialize to the same JSON text (given same | ||||
1573 | settings and version of Cpanel::JSON::XS), but this incurs a runtime overhead | ||||
1574 | and is only rarely useful, e.g. when you want to compare some JSON text | ||||
1575 | against another for equality. | ||||
1576 | |||||
1577 | =item array references | ||||
1578 | |||||
1579 | Perl array references become JSON arrays. | ||||
1580 | |||||
1581 | =item other references | ||||
1582 | |||||
1583 | Other unblessed references are generally not allowed and will cause an | ||||
1584 | exception to be thrown, except for references to the integers C<0> and | ||||
1585 | C<1>, which get turned into C<false> and C<true> atoms in JSON. | ||||
1586 | |||||
1587 | With the option C<allow_stringify>, you can ignore the exception and return | ||||
1588 | the stringification of the perl value. | ||||
1589 | |||||
1590 | With the option C<allow_unknown>, you can ignore the exception and | ||||
1591 | return C<null> instead. | ||||
1592 | |||||
1593 | encode_json [\"x"] # => cannot encode reference to scalar 'SCALAR(0x..)' | ||||
1594 | # unless the scalar is 0 or 1 | ||||
1595 | encode_json [\0, \1] # yields [false,true] | ||||
1596 | |||||
1597 | allow_stringify->encode_json [\"x"] # yields "x" unlike JSON::PP | ||||
1598 | allow_unknown->encode_json [\"x"] # yields null as in JSON::PP | ||||
1599 | |||||
1600 | =item Cpanel::JSON::XS::true, Cpanel::JSON::XS::false | ||||
1601 | |||||
1602 | These special values become JSON true and JSON false values, | ||||
1603 | respectively. You can also use C<\1> and C<\0> or C<!0> and C<!1> | ||||
1604 | directly if you want. | ||||
1605 | |||||
1606 | encode_json [Cpanel::JSON::XS::false, Cpanel::JSON::XS::true] # yields [false,true] | ||||
1607 | encode_json [!1, !0], [JSON_TYPE_BOOL, JSON_TYPE_BOOL] # yields [false,true] | ||||
1608 | |||||
1609 | eq/ne comparisons with true, false: | ||||
1610 | |||||
1611 | false is eq to the empty string or the string 'false' or the special | ||||
1612 | empty string C<!!0> or C<!1>, i.e. C<SV_NO>, or the numbers 0 or 0.0. | ||||
1613 | |||||
1614 | true is eq to the string 'true' or to the special string C<!0> | ||||
1615 | (i.e. C<SV_YES>) or to the numbers 1 or 1.0. | ||||
1616 | |||||
1617 | =item blessed objects | ||||
1618 | |||||
1619 | Blessed objects are not directly representable in JSON, but | ||||
1620 | C<Cpanel::JSON::XS> allows various optional ways of handling | ||||
1621 | objects. See L<OBJECT SERIALIZATION>, below, for details. | ||||
1622 | |||||
1623 | See the C<allow_blessed> and C<convert_blessed> methods on various | ||||
1624 | options on how to deal with this: basically, you can choose between | ||||
1625 | throwing an exception, encoding the reference as if it weren't | ||||
1626 | blessed, use the objects overloaded stringification method or provide | ||||
1627 | your own serializer method. | ||||
1628 | |||||
1629 | =item simple scalars | ||||
1630 | |||||
1631 | Simple Perl scalars (any scalar that is not a reference) are the most | ||||
1632 | difficult objects to encode: Cpanel::JSON::XS will encode undefined | ||||
1633 | scalars or inf/nan as JSON C<null> values and other scalars to either | ||||
1634 | number or string in non-deterministic way which may be affected or | ||||
1635 | changed by Perl version or any other loaded Perl module. | ||||
1636 | |||||
1637 | If you want to have stable and deterministic types in JSON encoder then | ||||
1638 | use L<Cpanel::JSON::XS::Type>. | ||||
1639 | |||||
1640 | Alternative way for deterministic types is to use C<type_all_string> | ||||
1641 | method when all perl scalars are encoded to JSON strings. | ||||
1642 | |||||
1643 | Non-deterministic behavior is following: scalars that have last been | ||||
1644 | used in a string context before encoding as JSON strings, and anything | ||||
1645 | else as number value: | ||||
1646 | |||||
1647 | # dump as number | ||||
1648 | encode_json [2] # yields [2] | ||||
1649 | encode_json [-3.0e17] # yields [-3e+17] | ||||
1650 | my $value = 5; encode_json [$value] # yields [5] | ||||
1651 | |||||
1652 | # used as string, but the two representations are for the same number | ||||
1653 | print $value; | ||||
1654 | encode_json [$value] # yields [5] | ||||
1655 | |||||
1656 | # used as different string (non-matching dual-var) | ||||
1657 | my $str = '0 but true'; | ||||
1658 | my $num = 1 + $str; | ||||
1659 | encode_json [$num, $str] # yields [1,"0 but true"] | ||||
1660 | |||||
1661 | # undef becomes null | ||||
1662 | encode_json [undef] # yields [null] | ||||
1663 | |||||
1664 | # inf or nan becomes null, unless you answered | ||||
1665 | # "Do you want to handle inf/nan as strings" with yes | ||||
1666 | encode_json [9**9**9] # yields [null] | ||||
1667 | |||||
1668 | You can force the type to be a JSON string by stringifying it: | ||||
1669 | |||||
1670 | my $x = 3.1; # some variable containing a number | ||||
1671 | "$x"; # stringified | ||||
1672 | $x .= ""; # another, more awkward way to stringify | ||||
1673 | print $x; # perl does it for you, too, quite often | ||||
1674 | |||||
1675 | You can force the type to be a JSON number by numifying it: | ||||
1676 | |||||
1677 | my $x = "3"; # some variable containing a string | ||||
1678 | $x += 0; # numify it, ensuring it will be dumped as a number | ||||
1679 | $x *= 1; # same thing, the choice is yours. | ||||
1680 | |||||
1681 | Note that numerical precision has the same meaning as under Perl (so | ||||
1682 | binary to decimal conversion follows the same rules as in Perl, which | ||||
1683 | can differ to other languages). Also, your perl interpreter might expose | ||||
1684 | extensions to the floating point numbers of your platform, such as | ||||
1685 | infinities or NaN's - these cannot be represented in JSON, and thus | ||||
1686 | null is returned instead. Optionally you can configure it to stringify | ||||
1687 | inf and nan values. | ||||
1688 | |||||
1689 | =back | ||||
1690 | |||||
1691 | =head2 OBJECT SERIALIZATION | ||||
1692 | |||||
1693 | As JSON cannot directly represent Perl objects, you have to choose between | ||||
1694 | a pure JSON representation (without the ability to deserialize the object | ||||
1695 | automatically again), and a nonstandard extension to the JSON syntax, | ||||
1696 | tagged values. | ||||
1697 | |||||
1698 | =head3 SERIALIZATION | ||||
1699 | |||||
1700 | What happens when C<Cpanel::JSON::XS> encounters a Perl object depends | ||||
1701 | on the C<allow_blessed>, C<convert_blessed> and C<allow_tags> | ||||
1702 | settings, which are used in this order: | ||||
1703 | |||||
1704 | =over 4 | ||||
1705 | |||||
1706 | =item 1. C<allow_tags> is enabled and the object has a C<FREEZE> method. | ||||
1707 | |||||
1708 | In this case, C<Cpanel::JSON::XS> uses the L<Types::Serialiser> object | ||||
1709 | serialization protocol to create a tagged JSON value, using a nonstandard | ||||
1710 | extension to the JSON syntax. | ||||
1711 | |||||
1712 | This works by invoking the C<FREEZE> method on the object, with the first | ||||
1713 | argument being the object to serialize, and the second argument being the | ||||
1714 | constant string C<JSON> to distinguish it from other serializers. | ||||
1715 | |||||
1716 | The C<FREEZE> method can return any number of values (i.e. zero or | ||||
1717 | more). These values and the paclkage/classname of the object will then be | ||||
1718 | encoded as a tagged JSON value in the following format: | ||||
1719 | |||||
1720 | ("classname")[FREEZE return values...] | ||||
1721 | |||||
1722 | e.g.: | ||||
1723 | |||||
1724 | ("URI")["http://www.google.com/"] | ||||
1725 | ("MyDate")[2013,10,29] | ||||
1726 | ("ImageData::JPEG")["Z3...VlCg=="] | ||||
1727 | |||||
1728 | For example, the hypothetical C<My::Object> C<FREEZE> method might use the | ||||
1729 | objects C<type> and C<id> members to encode the object: | ||||
1730 | |||||
1731 | sub My::Object::FREEZE { | ||||
1732 | my ($self, $serializer) = @_; | ||||
1733 | |||||
1734 | ($self->{type}, $self->{id}) | ||||
1735 | } | ||||
1736 | |||||
1737 | =item 2. C<convert_blessed> is enabled and the object has a C<TO_JSON> method. | ||||
1738 | |||||
1739 | In this case, the C<TO_JSON> method of the object is invoked in scalar | ||||
1740 | context. It must return a single scalar that can be directly encoded into | ||||
1741 | JSON. This scalar replaces the object in the JSON text. | ||||
1742 | |||||
1743 | For example, the following C<TO_JSON> method will convert all L<URI> | ||||
1744 | objects to JSON strings when serialized. The fact that these values | ||||
1745 | originally were L<URI> objects is lost. | ||||
1746 | |||||
1747 | sub URI::TO_JSON { | ||||
1748 | my ($uri) = @_; | ||||
1749 | $uri->as_string | ||||
1750 | } | ||||
1751 | |||||
1752 | =item 3. C<convert_blessed> is enabled and the object has a stringification overload. | ||||
1753 | |||||
1754 | In this case, the overloaded C<""> method of the object is invoked in scalar | ||||
1755 | context. It must return a single scalar that can be directly encoded into | ||||
1756 | JSON. This scalar replaces the object in the JSON text. | ||||
1757 | |||||
1758 | For example, the following C<""> method will convert all L<URI> | ||||
1759 | objects to JSON strings when serialized. The fact that these values | ||||
1760 | originally were L<URI> objects is lost. | ||||
1761 | |||||
1762 | package URI; | ||||
1763 | use overload '""' => sub { shift->as_string }; | ||||
1764 | |||||
1765 | =item 4. C<allow_blessed> is enabled. | ||||
1766 | |||||
1767 | The object will be serialized as a JSON null value. | ||||
1768 | |||||
1769 | =item 5. none of the above | ||||
1770 | |||||
1771 | If none of the settings are enabled or the respective methods are missing, | ||||
1772 | C<Cpanel::JSON::XS> throws an exception. | ||||
1773 | |||||
1774 | =back | ||||
1775 | |||||
1776 | =head3 DESERIALIZATION | ||||
1777 | |||||
1778 | For deserialization there are only two cases to consider: either | ||||
1779 | nonstandard tagging was used, in which case C<allow_tags> decides, | ||||
1780 | or objects cannot be automatically be deserialized, in which | ||||
1781 | case you can use postprocessing or the C<filter_json_object> or | ||||
1782 | C<filter_json_single_key_object> callbacks to get some real objects our of | ||||
1783 | your JSON. | ||||
1784 | |||||
1785 | This section only considers the tagged value case: I a tagged JSON object | ||||
1786 | is encountered during decoding and C<allow_tags> is disabled, a parse | ||||
1787 | error will result (as if tagged values were not part of the grammar). | ||||
1788 | |||||
1789 | If C<allow_tags> is enabled, C<Cpanel::JSON::XS> will look up the C<THAW> method | ||||
1790 | of the package/classname used during serialization (it will not attempt | ||||
1791 | to load the package as a Perl module). If there is no such method, the | ||||
1792 | decoding will fail with an error. | ||||
1793 | |||||
1794 | Otherwise, the C<THAW> method is invoked with the classname as first | ||||
1795 | argument, the constant string C<JSON> as second argument, and all the | ||||
1796 | values from the JSON array (the values originally returned by the | ||||
1797 | C<FREEZE> method) as remaining arguments. | ||||
1798 | |||||
1799 | The method must then return the object. While technically you can return | ||||
1800 | any Perl scalar, you might have to enable the C<enable_nonref> setting to | ||||
1801 | make that work in all cases, so better return an actual blessed reference. | ||||
1802 | |||||
1803 | As an example, let's implement a C<THAW> function that regenerates the | ||||
1804 | C<My::Object> from the C<FREEZE> example earlier: | ||||
1805 | |||||
1806 | sub My::Object::THAW { | ||||
1807 | my ($class, $serializer, $type, $id) = @_; | ||||
1808 | |||||
1809 | $class->new (type => $type, id => $id) | ||||
1810 | } | ||||
1811 | |||||
1812 | See the L</SECURITY CONSIDERATIONS> section below. Allowing external | ||||
1813 | json objects being deserialized to perl objects is usually a very bad | ||||
1814 | idea. | ||||
1815 | |||||
1816 | |||||
1817 | =head1 ENCODING/CODESET FLAG NOTES | ||||
1818 | |||||
1819 | The interested reader might have seen a number of flags that signify | ||||
1820 | encodings or codesets - C<utf8>, C<latin1>, C<binary> and | ||||
1821 | C<ascii>. There seems to be some confusion on what these do, so here | ||||
1822 | is a short comparison: | ||||
1823 | |||||
1824 | C<utf8> controls whether the JSON text created by C<encode> (and expected | ||||
1825 | by C<decode>) is UTF-8 encoded or not, while C<latin1> and C<ascii> only | ||||
1826 | control whether C<encode> escapes character values outside their respective | ||||
1827 | codeset range. Neither of these flags conflict with each other, although | ||||
1828 | some combinations make less sense than others. | ||||
1829 | |||||
1830 | Care has been taken to make all flags symmetrical with respect to | ||||
1831 | C<encode> and C<decode>, that is, texts encoded with any combination of | ||||
1832 | these flag values will be correctly decoded when the same flags are used | ||||
1833 | - in general, if you use different flag settings while encoding vs. when | ||||
1834 | decoding you likely have a bug somewhere. | ||||
1835 | |||||
1836 | Below comes a verbose discussion of these flags. Note that a "codeset" is | ||||
1837 | simply an abstract set of character-codepoint pairs, while an encoding | ||||
1838 | takes those codepoint numbers and I<encodes> them, in our case into | ||||
1839 | octets. Unicode is (among other things) a codeset, UTF-8 is an encoding, | ||||
1840 | and ISO-8859-1 (= latin 1) and ASCII are both codesets I<and> encodings at | ||||
1841 | the same time, which can be confusing. | ||||
1842 | |||||
1843 | =over 4 | ||||
1844 | |||||
1845 | =item C<utf8> flag disabled | ||||
1846 | |||||
1847 | When C<utf8> is disabled (the default), then C<encode>/C<decode> generate | ||||
1848 | and expect Unicode strings, that is, characters with high ordinal Unicode | ||||
1849 | values (> 255) will be encoded as such characters, and likewise such | ||||
1850 | characters are decoded as-is, no changes to them will be done, except | ||||
1851 | "(re-)interpreting" them as Unicode codepoints or Unicode characters, | ||||
1852 | respectively (to Perl, these are the same thing in strings unless you do | ||||
1853 | funny/weird/dumb stuff). | ||||
1854 | |||||
1855 | This is useful when you want to do the encoding yourself (e.g. when you | ||||
1856 | want to have UTF-16 encoded JSON texts) or when some other layer does | ||||
1857 | the encoding for you (for example, when printing to a terminal using a | ||||
1858 | filehandle that transparently encodes to UTF-8 you certainly do NOT want | ||||
1859 | to UTF-8 encode your data first and have Perl encode it another time). | ||||
1860 | |||||
1861 | =item C<utf8> flag enabled | ||||
1862 | |||||
1863 | If the C<utf8>-flag is enabled, C<encode>/C<decode> will encode all | ||||
1864 | characters using the corresponding UTF-8 multi-byte sequence, and will | ||||
1865 | expect your input strings to be encoded as UTF-8, that is, no "character" | ||||
1866 | of the input string must have any value > 255, as UTF-8 does not allow | ||||
1867 | that. | ||||
1868 | |||||
1869 | The C<utf8> flag therefore switches between two modes: disabled means you | ||||
1870 | will get a Unicode string in Perl, enabled means you get an UTF-8 encoded | ||||
1871 | octet/binary string in Perl. | ||||
1872 | |||||
1873 | =item C<latin1>, C<binary> or C<ascii> flags enabled | ||||
1874 | |||||
1875 | With C<latin1> (or C<ascii>) enabled, C<encode> will escape | ||||
1876 | characters with ordinal values > 255 (> 127 with C<ascii>) and encode | ||||
1877 | the remaining characters as specified by the C<utf8> flag. | ||||
1878 | With C<binary> enabled, ordinal values > 255 are illegal. | ||||
1879 | |||||
1880 | If C<utf8> is disabled, then the result is also correctly encoded in those | ||||
1881 | character sets (as both are proper subsets of Unicode, meaning that a | ||||
1882 | Unicode string with all character values < 256 is the same thing as a | ||||
1883 | ISO-8859-1 string, and a Unicode string with all character values < 128 is | ||||
1884 | the same thing as an ASCII string in Perl). | ||||
1885 | |||||
1886 | If C<utf8> is enabled, you still get a correct UTF-8-encoded string, | ||||
1887 | regardless of these flags, just some more characters will be escaped using | ||||
1888 | C<\uXXXX> then before. | ||||
1889 | |||||
1890 | Note that ISO-8859-1-I<encoded> strings are not compatible with UTF-8 | ||||
1891 | encoding, while ASCII-encoded strings are. That is because the ISO-8859-1 | ||||
1892 | encoding is NOT a subset of UTF-8 (despite the ISO-8859-1 I<codeset> being | ||||
1893 | a subset of Unicode), while ASCII is. | ||||
1894 | |||||
1895 | Surprisingly, C<decode> will ignore these flags and so treat all input | ||||
1896 | values as governed by the C<utf8> flag. If it is disabled, this allows you | ||||
1897 | to decode ISO-8859-1- and ASCII-encoded strings, as both strict subsets of | ||||
1898 | Unicode. If it is enabled, you can correctly decode UTF-8 encoded strings. | ||||
1899 | |||||
1900 | So neither C<latin1>, C<binary> nor C<ascii> are incompatible with the | ||||
1901 | C<utf8> flag - they only govern when the JSON output engine escapes a | ||||
1902 | character or not. | ||||
1903 | |||||
1904 | The main use for C<latin1> or C<binary> is to relatively efficiently | ||||
1905 | store binary data as JSON, at the expense of breaking compatibility | ||||
1906 | with most JSON decoders. | ||||
1907 | |||||
1908 | The main use for C<ascii> is to force the output to not contain characters | ||||
1909 | with values > 127, which means you can interpret the resulting string | ||||
1910 | as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and | ||||
1911 | 8-bit-encoding, and still get the same data structure back. This is useful | ||||
1912 | when your channel for JSON transfer is not 8-bit clean or the encoding | ||||
1913 | might be mangled in between (e.g. in mail), and works because ASCII is a | ||||
1914 | proper subset of most 8-bit and multibyte encodings in use in the world. | ||||
1915 | |||||
1916 | =back | ||||
1917 | |||||
1918 | |||||
1919 | =head2 JSON and ECMAscript | ||||
1920 | |||||
1921 | JSON syntax is based on how literals are represented in javascript (the | ||||
1922 | not-standardized predecessor of ECMAscript) which is presumably why it is | ||||
1923 | called "JavaScript Object Notation". | ||||
1924 | |||||
1925 | However, JSON is not a subset (and also not a superset of course) of | ||||
1926 | ECMAscript (the standard) or javascript (whatever browsers actually | ||||
1927 | implement). | ||||
1928 | |||||
1929 | If you want to use javascript's C<eval> function to "parse" JSON, you | ||||
1930 | might run into parse errors for valid JSON texts, or the resulting data | ||||
1931 | structure might not be queryable: | ||||
1932 | |||||
1933 | One of the problems is that U+2028 and U+2029 are valid characters inside | ||||
1934 | JSON strings, but are not allowed in ECMAscript string literals, so the | ||||
1935 | following Perl fragment will not output something that can be guaranteed | ||||
1936 | to be parsable by javascript's C<eval>: | ||||
1937 | |||||
1938 | use Cpanel::JSON::XS; | ||||
1939 | |||||
1940 | print encode_json [chr 0x2028]; | ||||
1941 | |||||
1942 | The right fix for this is to use a proper JSON parser in your javascript | ||||
1943 | programs, and not rely on C<eval> (see for example Douglas Crockford's | ||||
1944 | F<json2.js> parser). | ||||
1945 | |||||
1946 | If this is not an option, you can, as a stop-gap measure, simply encode to | ||||
1947 | ASCII-only JSON: | ||||
1948 | |||||
1949 | use Cpanel::JSON::XS; | ||||
1950 | |||||
1951 | print Cpanel::JSON::XS->new->ascii->encode ([chr 0x2028]); | ||||
1952 | |||||
1953 | Note that this will enlarge the resulting JSON text quite a bit if you | ||||
1954 | have many non-ASCII characters. You might be tempted to run some regexes | ||||
1955 | to only escape U+2028 and U+2029, e.g.: | ||||
1956 | |||||
1957 | # DO NOT USE THIS! | ||||
1958 | my $json = Cpanel::JSON::XS->new->utf8->encode ([chr 0x2028]); | ||||
1959 | $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028 | ||||
1960 | $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029 | ||||
1961 | print $json; | ||||
1962 | |||||
1963 | Note that I<this is a bad idea>: the above only works for U+2028 and | ||||
1964 | U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing | ||||
1965 | javascript implementations, however, have issues with other characters as | ||||
1966 | well - using C<eval> naively simply I<will> cause problems. | ||||
1967 | |||||
1968 | Another problem is that some javascript implementations reserve | ||||
1969 | some property names for their own purposes (which probably makes | ||||
1970 | them non-ECMAscript-compliant). For example, Iceweasel reserves the | ||||
1971 | C<__proto__> property name for its own purposes. | ||||
1972 | |||||
1973 | If that is a problem, you could parse try to filter the resulting JSON | ||||
1974 | output for these property strings, e.g.: | ||||
1975 | |||||
1976 | $json =~ s/"__proto__"\s*:/"__proto__renamed":/g; | ||||
1977 | |||||
1978 | This works because C<__proto__> is not valid outside of strings, so every | ||||
1979 | occurrence of C<"__proto__"\s*:> must be a string used as property name. | ||||
1980 | |||||
1981 | Unicode non-characters between U+FFFD and U+10FFFF are decoded either | ||||
1982 | to the recommended U+FFFD REPLACEMENT CHARACTER (see Unicode PR #121: | ||||
1983 | Recommended Practice for Replacement Characters), or in the binary or | ||||
1984 | relaxed mode left as is, keeping the illegal non-characters as before. | ||||
1985 | |||||
1986 | Raw non-Unicode characters outside the valid unicode range fail now to | ||||
1987 | parse, because "A string is a sequence of zero or more Unicode | ||||
1988 | characters" RFC 7159 section 1 and "JSON text SHALL be encoded in | ||||
1989 | Unicode RFC 7159 section 8.1. We use now the UTF8_DISALLOW_SUPER | ||||
1990 | flag when parsing unicode. | ||||
1991 | |||||
1992 | If you know of other incompatibilities, please let me know. | ||||
1993 | |||||
1994 | |||||
1995 | =head2 JSON and YAML | ||||
1996 | |||||
1997 | You often hear that JSON is a subset of YAML. I<in general, there is | ||||
1998 | no way to configure JSON::XS to output a data structure as valid YAML> | ||||
1999 | that works in all cases. If you really must use Cpanel::JSON::XS to | ||||
2000 | generate YAML, you should use this algorithm (subject to change in | ||||
2001 | future versions): | ||||
2002 | |||||
2003 | my $to_yaml = Cpanel::JSON::XS->new->utf8->space_after (1); | ||||
2004 | my $yaml = $to_yaml->encode ($ref) . "\n"; | ||||
2005 | |||||
2006 | This will I<usually> generate JSON texts that also parse as valid | ||||
2007 | YAML. | ||||
2008 | |||||
2009 | |||||
2010 | =head2 SPEED | ||||
2011 | |||||
2012 | It seems that JSON::XS is surprisingly fast, as shown in the following | ||||
2013 | tables. They have been generated with the help of the C<eg/bench> program | ||||
2014 | in the JSON::XS distribution, to make it easy to compare on your own | ||||
2015 | system. | ||||
2016 | |||||
2017 | JSON::XS is with L<Data::MessagePack> and L<Sereal> one of the fastest | ||||
2018 | serializers, because JSON and JSON::XS do not support backrefs (no | ||||
2019 | graph structures), only trees. Storable supports backrefs, | ||||
2020 | i.e. graphs. Data::MessagePack encodes its data binary (as Storable) | ||||
2021 | and supports only very simple subset of JSON. | ||||
2022 | |||||
2023 | First comes a comparison between various modules using | ||||
2024 | a very short single-line JSON string (also available at | ||||
2025 | L<http://dist.schmorp.de/misc/json/short.json>). | ||||
2026 | |||||
2027 | {"method": "handleMessage", "params": ["user1", | ||||
2028 | "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, | ||||
2029 | 1, 0]} | ||||
2030 | |||||
2031 | It shows the number of encodes/decodes per second (JSON::XS uses | ||||
2032 | the functional interface, while Cpanel::JSON::XS/2 uses the OO interface | ||||
2033 | with pretty-printing and hash key sorting enabled, Cpanel::JSON::XS/3 enables | ||||
2034 | shrink. JSON::DWIW/DS uses the deserialize function, while JSON::DWIW::FJ | ||||
2035 | uses the from_json method). Higher is better: | ||||
2036 | |||||
2037 | module | encode | decode | | ||||
2038 | --------------|------------|------------| | ||||
2039 | JSON::DWIW/DS | 86302.551 | 102300.098 | | ||||
2040 | JSON::DWIW/FJ | 86302.551 | 75983.768 | | ||||
2041 | JSON::PP | 15827.562 | 6638.658 | | ||||
2042 | JSON::Syck | 63358.066 | 47662.545 | | ||||
2043 | JSON::XS | 511500.488 | 511500.488 | | ||||
2044 | JSON::XS/2 | 291271.111 | 388361.481 | | ||||
2045 | JSON::XS/3 | 361577.931 | 361577.931 | | ||||
2046 | Storable | 66788.280 | 265462.278 | | ||||
2047 | --------------+------------+------------+ | ||||
2048 | |||||
2049 | That is, JSON::XS is almost six times faster than JSON::DWIW on encoding, | ||||
2050 | about five times faster on decoding, and over thirty to seventy times | ||||
2051 | faster than JSON's pure perl implementation. It also compares favourably | ||||
2052 | to Storable for small amounts of data. | ||||
2053 | |||||
2054 | Using a longer test string (roughly 18KB, generated from Yahoo! Locals | ||||
2055 | search API (L<http://dist.schmorp.de/misc/json/long.json>). | ||||
2056 | |||||
2057 | module | encode | decode | | ||||
2058 | --------------|------------|------------| | ||||
2059 | JSON::DWIW/DS | 1647.927 | 2673.916 | | ||||
2060 | JSON::DWIW/FJ | 1630.249 | 2596.128 | | ||||
2061 | JSON::PP | 400.640 | 62.311 | | ||||
2062 | JSON::Syck | 1481.040 | 1524.869 | | ||||
2063 | JSON::XS | 20661.596 | 9541.183 | | ||||
2064 | JSON::XS/2 | 10683.403 | 9416.938 | | ||||
2065 | JSON::XS/3 | 20661.596 | 9400.054 | | ||||
2066 | Storable | 19765.806 | 10000.725 | | ||||
2067 | --------------+------------+------------+ | ||||
2068 | |||||
2069 | Again, JSON::XS leads by far (except for Storable which non-surprisingly | ||||
2070 | decodes a bit faster). | ||||
2071 | |||||
2072 | On large strings containing lots of high Unicode characters, some modules | ||||
2073 | (such as JSON::PC) seem to decode faster than JSON::XS, but the result | ||||
2074 | will be broken due to missing (or wrong) Unicode handling. Others refuse | ||||
2075 | to decode or encode properly, so it was impossible to prepare a fair | ||||
2076 | comparison table for that case. | ||||
2077 | |||||
2078 | For updated graphs see L<https://github.com/Sereal/Sereal/wiki/Sereal-Comparison-Graphs> | ||||
2079 | |||||
2080 | |||||
2081 | =head1 INTEROP with JSON and JSON::XS and other JSON modules | ||||
2082 | |||||
2083 | As long as you only serialize data that can be directly expressed in | ||||
2084 | JSON, C<Cpanel::JSON::XS> is incapable of generating invalid JSON | ||||
2085 | output (modulo bugs, but C<JSON::XS> has found more bugs in the | ||||
2086 | official JSON testsuite (1) than the official JSON testsuite has found | ||||
2087 | in C<JSON::XS> (0)). | ||||
2088 | C<Cpanel::JSON::XS> is currently the only known JSON decoder which passes all | ||||
2089 | L<http://seriot.ch/parsing_json.html> tests, while being the fastest also. | ||||
2090 | |||||
2091 | When you have trouble decoding JSON generated by this module using other | ||||
2092 | decoders, then it is very likely that you have an encoding mismatch or the | ||||
2093 | other decoder is broken. | ||||
2094 | |||||
2095 | When decoding, C<JSON::XS> is strict by default and will likely catch | ||||
2096 | all errors. There are currently two settings that change this: | ||||
2097 | C<relaxed> makes C<JSON::XS> accept (but not generate) some | ||||
2098 | non-standard extensions, and C<allow_tags> or C<allow_blessed> will | ||||
2099 | allow you to encode and decode Perl objects, at the cost of being | ||||
2100 | totally insecure and not outputting valid JSON anymore. | ||||
2101 | |||||
2102 | JSON-XS-3.01 broke interoperability with JSON-2.90 with booleans. See L<JSON>. | ||||
2103 | |||||
2104 | Cpanel::JSON::XS needs to know the JSON and JSON::XS versions to be able work | ||||
2105 | with those objects, especially when encoding a booleans like C<{"is_true":true}>. | ||||
2106 | So you need to load these modules before. | ||||
2107 | |||||
2108 | true/false overloading and boolean representations are supported. | ||||
2109 | |||||
2110 | JSON::XS and JSON::PP representations are accepted and older JSON::XS | ||||
2111 | accepts Cpanel::JSON::XS booleans. All JSON modules JSON, JSON, PP, | ||||
2112 | JSON::XS, Cpanel::JSON::XS produce JSON::PP::Boolean objects, just | ||||
2113 | Mojo and JSON::YAJL not. Mojo produces Mojo::JSON::_Bool and | ||||
2114 | JSON::YAJL::Parser just an unblessed IV. | ||||
2115 | |||||
2116 | Cpanel::JSON::XS accepts JSON::PP::Boolean and Mojo::JSON::_Bool | ||||
2117 | objects as booleans. | ||||
2118 | |||||
2119 | I cannot think of any reason to still use JSON::XS anymore. | ||||
2120 | |||||
2121 | |||||
2122 | =head2 TAGGED VALUE SYNTAX AND STANDARD JSON EN/DECODERS | ||||
2123 | |||||
2124 | When you use C<allow_tags> to use the extended (and also nonstandard | ||||
2125 | and invalid) JSON syntax for serialized objects, and you still want to | ||||
2126 | decode the generated serialize objects, you can run a regex to replace | ||||
2127 | the tagged syntax by standard JSON arrays (it only works for "normal" | ||||
2128 | package names without comma, newlines or single colons). First, the | ||||
2129 | readable Perl version: | ||||
2130 | |||||
2131 | # if your FREEZE methods return no values, you need this replace first: | ||||
2132 | $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx; | ||||
2133 | |||||
2134 | # this works for non-empty constructor arg lists: | ||||
2135 | $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[/[$1,/gx; | ||||
2136 | |||||
2137 | And here is a less readable version that is easy to adapt to other | ||||
2138 | languages: | ||||
2139 | |||||
2140 | $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/[$1,/g; | ||||
2141 | |||||
2142 | Here is an ECMAScript version (same regex): | ||||
2143 | |||||
2144 | json = json.replace (/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/g, "[$1,"); | ||||
2145 | |||||
2146 | Since this syntax converts to standard JSON arrays, it might be hard to | ||||
2147 | distinguish serialized objects from normal arrays. You can prepend a | ||||
2148 | "magic number" as first array element to reduce chances of a collision: | ||||
2149 | |||||
2150 | $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/["XU1peReLzT4ggEllLanBYq4G9VzliwKF",$1,/g; | ||||
2151 | |||||
2152 | And after decoding the JSON text, you could walk the data | ||||
2153 | structure looking for arrays with a first element of | ||||
2154 | C<XU1peReLzT4ggEllLanBYq4G9VzliwKF>. | ||||
2155 | |||||
2156 | The same approach can be used to create the tagged format with another | ||||
2157 | encoder. First, you create an array with the magic string as first member, | ||||
2158 | the classname as second, and constructor arguments last, encode it as part | ||||
2159 | of your JSON structure, and then: | ||||
2160 | |||||
2161 | $json =~ s/\[\s*"XU1peReLzT4ggEllLanBYq4G9VzliwKF"\s*,\s*("([^\\":,]+|\\.|::)*")\s*,/($1)[/g; | ||||
2162 | |||||
2163 | Again, this has some limitations - the magic string must not be encoded | ||||
2164 | with character escapes, and the constructor arguments must be non-empty. | ||||
2165 | |||||
2166 | |||||
2167 | =head1 RFC7159 | ||||
2168 | |||||
2169 | Since this module was written, Google has written a new JSON RFC, RFC 7159 | ||||
2170 | (and RFC7158). Unfortunately, this RFC breaks compatibility with both the | ||||
2171 | original JSON specification on www.json.org and RFC4627. | ||||
2172 | |||||
2173 | As far as I can see, you can get partial compatibility when parsing by | ||||
2174 | using C<< ->allow_nonref >>. However, consider the security implications | ||||
2175 | of doing so. | ||||
2176 | |||||
2177 | I haven't decided yet when to break compatibility with RFC4627 by default | ||||
2178 | (and potentially leave applications insecure) and change the default to | ||||
2179 | follow RFC7159, but application authors are well advised to call C<< | ||||
2180 | ->allow_nonref(0) >> even if this is the current default, if they cannot | ||||
2181 | handle non-reference values, in preparation for the day when the default | ||||
2182 | will change. | ||||
2183 | |||||
2184 | =head1 SECURITY CONSIDERATIONS | ||||
2185 | |||||
2186 | JSON::XS and Cpanel::JSON::XS are not only fast. JSON is generally the | ||||
2187 | most secure serializing format, because it is the only one besides | ||||
2188 | Data::MessagePack, which does not deserialize objects per default. For | ||||
2189 | all languages, not just perl. The binary variant BSON (MongoDB) does | ||||
2190 | more but is unsafe. | ||||
2191 | |||||
2192 | It is trivial for any attacker to create such serialized objects in | ||||
2193 | JSON and trick perl into expanding them, thereby triggering certain | ||||
2194 | methods. Watch L<https://www.youtube.com/watch?v=Gzx6KlqiIZE> for an | ||||
2195 | exploit demo for "CVE-2015-1592 SixApart MovableType Storable Perl | ||||
2196 | Code Execution" for a deserializer which expands objects. | ||||
2197 | Deserializing even coderefs (methods, functions) or external | ||||
2198 | data would be considered the most dangerous. | ||||
2199 | |||||
2200 | Security relevant overview of serializers regarding deserializing | ||||
2201 | objects by default: | ||||
2202 | |||||
2203 | Objects Coderefs External Data | ||||
2204 | |||||
2205 | Data::Dumper YES YES YES | ||||
2206 | Storable YES NO (def) NO | ||||
2207 | Sereal YES NO NO | ||||
2208 | YAML YES NO NO | ||||
2209 | B::C YES YES YES | ||||
2210 | B::Bytecode YES YES YES | ||||
2211 | BSON YES YES NO | ||||
2212 | JSON::SL YES NO YES | ||||
2213 | JSON NO (def) NO NO | ||||
2214 | Data::MessagePack NO NO NO | ||||
2215 | XML NO NO YES | ||||
2216 | |||||
2217 | Pickle YES YES YES | ||||
2218 | PHP Deserialize YES NO NO | ||||
2219 | |||||
2220 | When you are using JSON in a protocol, talking to untrusted potentially | ||||
2221 | hostile creatures requires relatively few measures. | ||||
2222 | |||||
2223 | First of all, your JSON decoder should be secure, that is, should not have | ||||
2224 | any buffer overflows. Obviously, this module should ensure that. | ||||
2225 | |||||
2226 | Second, you need to avoid resource-starving attacks. That means you should | ||||
2227 | limit the size of JSON texts you accept, or make sure then when your | ||||
2228 | resources run out, that's just fine (e.g. by using a separate process that | ||||
2229 | can crash safely). The size of a JSON text in octets or characters is | ||||
2230 | usually a good indication of the size of the resources required to decode | ||||
2231 | it into a Perl structure. While JSON::XS can check the size of the JSON | ||||
2232 | text, it might be too late when you already have it in memory, so you | ||||
2233 | might want to check the size before you accept the string. | ||||
2234 | |||||
2235 | Third, Cpanel::JSON::XS recurses using the C stack when decoding objects and | ||||
2236 | arrays. The C stack is a limited resource: for instance, on my amd64 | ||||
2237 | machine with 8MB of stack size I can decode around 180k nested arrays but | ||||
2238 | only 14k nested JSON objects (due to perl itself recursing deeply on croak | ||||
2239 | to free the temporary). If that is exceeded, the program crashes. To be | ||||
2240 | conservative, the default nesting limit is set to 512. If your process | ||||
2241 | has a smaller stack, you should adjust this setting accordingly with the | ||||
2242 | C<max_depth> method. | ||||
2243 | |||||
2244 | Also keep in mind that Cpanel::JSON::XS might leak contents of your Perl data | ||||
2245 | structures in its error messages, so when you serialize sensitive | ||||
2246 | information you might want to make sure that exceptions thrown by JSON::XS | ||||
2247 | will not end up in front of untrusted eyes. | ||||
2248 | |||||
2249 | If you are using Cpanel::JSON::XS to return packets to consumption | ||||
2250 | by JavaScript scripts in a browser you should have a look at | ||||
2251 | L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to | ||||
2252 | see whether you are vulnerable to some common attack vectors (which really | ||||
2253 | are browser design bugs, but it is still you who will have to deal with | ||||
2254 | it, as major browser developers care only for features, not about getting | ||||
2255 | security right). You might also want to also look at L<Mojo::JSON> | ||||
2256 | special escape rules to prevent from XSS attacks. | ||||
2257 | |||||
2258 | =head1 "OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159) | ||||
2259 | |||||
2260 | TL;DR: Due to security concerns, Cpanel::JSON::XS will not allow | ||||
2261 | scalar data in JSON texts by default - you need to create your own | ||||
2262 | Cpanel::JSON::XS object and enable C<allow_nonref>: | ||||
2263 | |||||
2264 | |||||
2265 | my $json = JSON::XS->new->allow_nonref; | ||||
2266 | |||||
2267 | $text = $json->encode ($data); | ||||
2268 | $data = $json->decode ($text); | ||||
2269 | |||||
2270 | The long version: JSON being an important and supposedly stable format, | ||||
2271 | the IETF standardized it as RFC 4627 in 2006. Unfortunately the inventor | ||||
2272 | of JSON Douglas Crockford unilaterally changed the definition of JSON in | ||||
2273 | javascript. Rather than create a fork, the IETF decided to standardize the | ||||
2274 | new syntax (apparently, so I as told, without finding it very amusing). | ||||
2275 | |||||
2276 | The biggest difference between the original JSON and the new JSON is that | ||||
2277 | the new JSON supports scalars (anything other than arrays and objects) at | ||||
2278 | the top-level of a JSON text. While this is strictly backwards compatible | ||||
2279 | to older versions, it breaks a number of protocols that relied on sending | ||||
2280 | JSON back-to-back, and is a minor security concern. | ||||
2281 | |||||
2282 | For example, imagine you have two banks communicating, and on one side, | ||||
2283 | the JSON coder gets upgraded. Two messages, such as C<10> and C<1000> | ||||
2284 | might then be confused to mean C<101000>, something that couldn't happen | ||||
2285 | in the original JSON, because neither of these messages would be valid | ||||
2286 | JSON. | ||||
2287 | |||||
2288 | If one side accepts these messages, then an upgrade in the coder on either | ||||
2289 | side could result in this becoming exploitable. | ||||
2290 | |||||
2291 | This module has always allowed these messages as an optional extension, by | ||||
2292 | default disabled. The security concerns are the reason why the default is | ||||
2293 | still disabled, but future versions might/will likely upgrade to the newer | ||||
2294 | RFC as default format, so you are advised to check your implementation | ||||
2295 | and/or override the default with C<< ->allow_nonref (0) >> to ensure that | ||||
2296 | future versions are safe. | ||||
2297 | |||||
2298 | =head1 THREADS | ||||
2299 | |||||
2300 | Cpanel::JSON::XS has proper ithreads support, unlike JSON::XS. If you | ||||
2301 | encounter any bugs with thread support please report them. | ||||
2302 | |||||
2303 | From Version 4.00 - 4.19 you couldn't encode true with threads::shared | ||||
2304 | magic. | ||||
2305 | |||||
2306 | =head1 BUGS | ||||
2307 | |||||
2308 | While the goal of the Cpanel::JSON::XS module is to be correct, that | ||||
2309 | unfortunately does not mean it's bug-free, only that the author thinks | ||||
2310 | its design is bug-free. If you keep reporting bugs and tests they will | ||||
2311 | be fixed swiftly, though. | ||||
2312 | |||||
2313 | Since the JSON::XS author refuses to use a public bugtracker and | ||||
2314 | prefers private emails, we use the tracker at B<github>, so you might want | ||||
2315 | to report any issues twice. Once in private to MLEHMANN to be fixed in | ||||
2316 | JSON::XS and one to our the public tracker. Issues fixed by JSON::XS | ||||
2317 | with a new release will also be backported to Cpanel::JSON::XS and | ||||
2318 | 5.6.2, as long as cPanel relies on 5.6.2 and Cpanel::JSON::XS as our | ||||
2319 | serializer of choice. | ||||
2320 | |||||
2321 | L<https://github.com/rurban/Cpanel-JSON-XS/issues> | ||||
2322 | |||||
2323 | =head1 LICENSE | ||||
2324 | |||||
2325 | This module is available under the same licences as perl, the Artistic | ||||
2326 | license and the GPL. | ||||
2327 | |||||
2328 | =cut | ||||
2329 | |||||
2330 | sub allow_bigint { | ||||
2331 | Carp::carp("allow_bigint() is obsoleted. use allow_bignum() instead."); | ||||
2332 | } | ||||
2333 | |||||
2334 | # spent 2.32ms (2.04+282µs) within Cpanel::JSON::XS::BEGIN@2334 which was called:
# once (2.04ms+282µs) by JSON::MaybeXS::_choose_json_module at line 2372 | ||||
2335 | package | ||||
2336 | JSON::PP::Boolean; | ||||
2337 | |||||
2338 | 1 | 660µs | require overload; | ||
2339 | |||||
2340 | 1 | 3µs | local $^W; # silence redefine warnings. no warnings 'redefine' does not help | ||
2341 | # These already come with JSON::PP::Boolean. Avoid redefine warning. | ||||
2342 | 1 | 8µs | 1 | 37µs | if (!defined $JSON::PP::Boolean::VERSION or $JSON::PP::VERSION lt '4.00') { # spent 37µs making 1 call to overload::import |
2343 | &overload::import( 'overload', | ||||
2344 | "0+" => sub { ${$_[0]} }, | ||||
2345 | "++" => sub { $_[0] = ${$_[0]} + 1 }, | ||||
2346 | "--" => sub { $_[0] = ${$_[0]} - 1 }, | ||||
2347 | ); | ||||
2348 | } | ||||
2349 | # workaround 5.6 reserved keyword warning | ||||
2350 | &overload::import( 'overload', | ||||
2351 | '""' => sub { ${$_[0]} == 1 ? '1' : '0' }, # GH 29 | ||||
2352 | 'eq' => sub { | ||||
2353 | my ($obj, $op) = $_[2] ? ($_[1], $_[0]) : ($_[0], $_[1]); | ||||
2354 | #warn "eq obj:$obj op:$op len:", length($op) > 0, " swap:$_[2]"; | ||||
2355 | if (ref $op) { # if 2nd also blessed might recurse endlessly | ||||
2356 | return $obj ? 1 == $op : 0 == $op; | ||||
2357 | } | ||||
2358 | # if string, only accept numbers or true|false or "" (e.g. !!0 / SV_NO) | ||||
2359 | elsif ($op !~ /^[0-9]+$/) { | ||||
2360 | return "$obj" eq '1' ? 'true' eq $op : 'false' eq $op || "" eq $op; | ||||
2361 | } | ||||
2362 | else { | ||||
2363 | return $obj ? 1 == $op : 0 == $op; | ||||
2364 | } | ||||
2365 | }, | ||||
2366 | 'ne' => sub { | ||||
2367 | my ($obj, $op) = $_[2] ? ($_[1], $_[0]) : ($_[0], $_[1]); | ||||
2368 | #warn "ne obj:$obj op:$op"; | ||||
2369 | return !($obj eq $op); | ||||
2370 | }, | ||||
2371 | 1 | 9µs | 1 | 18µs | fallback => 1); # spent 18µs making 1 call to overload::import |
2372 | 1 | 93µs | 1 | 2.32ms | } # spent 2.32ms making 1 call to Cpanel::JSON::XS::BEGIN@2334 |
2373 | |||||
2374 | our ($true, $false); | ||||
2375 | # spent 7µs within Cpanel::JSON::XS::BEGIN@2375 which was called:
# once (7µs+0s) by JSON::MaybeXS::_choose_json_module at line 2385 | ||||
2376 | 1 | 5µs | if ($INC{'JSON/XS.pm'} | ||
2377 | and $INC{'Types/Serialiser.pm'} | ||||
2378 | and $JSON::XS::VERSION ge "3.00") { | ||||
2379 | $true = $Types::Serialiser::true; # readonly if loaded by JSON::XS | ||||
2380 | $false = $Types::Serialiser::false; | ||||
2381 | } else { | ||||
2382 | 1 | 1µs | $true = do { bless \(my $dummy = 1), "JSON::PP::Boolean" }; | ||
2383 | 1 | 0s | $false = do { bless \(my $dummy = 0), "JSON::PP::Boolean" }; | ||
2384 | } | ||||
2385 | 1 | 65µs | 1 | 7µs | } # spent 7µs making 1 call to Cpanel::JSON::XS::BEGIN@2375 |
2386 | |||||
2387 | # spent 8µs within Cpanel::JSON::XS::BEGIN@2387 which was called:
# once (8µs+0s) by JSON::MaybeXS::_choose_json_module at line 2392 | ||||
2388 | 1 | 0s | my $const_true = $true; | ||
2389 | 1 | 0s | my $const_false = $false; | ||
2390 | 1 | 4µs | *true = sub () { $const_true }; | ||
2391 | 1 | 5µs | *false = sub () { $const_false }; | ||
2392 | 1 | 135µs | 1 | 8µs | } # spent 8µs making 1 call to Cpanel::JSON::XS::BEGIN@2387 |
2393 | |||||
2394 | sub is_bool($) { | ||||
2395 | shift if @_ == 2; # as method call | ||||
2396 | (ref($_[0]) and UNIVERSAL::isa( $_[0], JSON::PP::Boolean::)) | ||||
2397 | or (exists $INC{'Types/Serialiser.pm'} and Types::Serialiser::is_bool($_[0])) | ||||
2398 | } | ||||
2399 | |||||
2400 | 1 | 1.56ms | 1 | 4.53ms | XSLoader::load 'Cpanel::JSON::XS', $XS_VERSION; # spent 4.53ms making 1 call to XSLoader::load |
2401 | |||||
2402 | 1 | 13µs | 1; | ||
2403 | |||||
2404 | =head1 SEE ALSO | ||||
2405 | |||||
2406 | The F<cpanel_json_xs> command line utility for quick experiments. | ||||
2407 | |||||
2408 | L<JSON>, L<JSON::XS>, L<JSON::MaybeXS>, L<Mojo::JSON>, L<Mojo::JSON::MaybeXS>, | ||||
2409 | L<JSON::SL>, L<JSON::DWIW>, L<JSON::YAJL>, L<JSON::Any>, L<Test::JSON>, | ||||
2410 | L<Locale::Wolowitz>, | ||||
2411 | L<https://metacpan.org/search?q=JSON> | ||||
2412 | |||||
2413 | L<https://tools.ietf.org/html/rfc7159> | ||||
2414 | |||||
2415 | L<https://tools.ietf.org/html/rfc4627> | ||||
2416 | |||||
2417 | |||||
2418 | =head1 AUTHOR | ||||
2419 | |||||
2420 | Reini Urban <rurban@cpan.org> | ||||
2421 | |||||
2422 | Marc Lehmann <schmorp@schmorp.de>, http://home.schmorp.de/ | ||||
2423 | |||||
2424 | =head1 MAINTAINER | ||||
2425 | |||||
2426 | Reini Urban <rurban@cpan.org> | ||||
2427 | |||||
2428 | =cut | ||||
2429 | |||||
# spent 480µs within Cpanel::JSON::XS::DESTROY which was called:
# once (480µs+0s) by main::NULL at line 0 of /Users/ether/git/fastly/Heavenly/profiling/jsm-0.552-om-0.031/../prof.pl | |||||
# spent 19µs within Cpanel::JSON::XS::END which was called:
# once (19µs+0s) by main::RUNTIME at line 0 of /Users/ether/git/fastly/Heavenly/profiling/jsm-0.552-om-0.031/../prof.pl | |||||
# spent 1µs within Cpanel::JSON::XS::__ANON__ which was called:
# once (1µs+0s) by Cpanel::JSON::XS::BEGIN@236 at line 236 | |||||
# spent 0s within Cpanel::JSON::XS::allow_bignum which was called:
# once (0s+0s) by JSON::MaybeXS::new at line 50 of JSON/MaybeXS.pm | |||||
# spent 1µs within Cpanel::JSON::XS::allow_nonref which was called:
# once (1µs+0s) by JSON::MaybeXS::new at line 50 of JSON/MaybeXS.pm | |||||
# spent 1µs within Cpanel::JSON::XS::canonical which was called:
# once (1µs+0s) by JSON::MaybeXS::new at line 50 of JSON/MaybeXS.pm | |||||
# spent 0s within Cpanel::JSON::XS::convert_blessed which was called:
# once (0s+0s) by JSON::MaybeXS::new at line 50 of JSON/MaybeXS.pm | |||||
# spent 1.41ms within Cpanel::JSON::XS::decode which was called 14 times, avg 101µs/call:
# 8 times (282µs+0s) by JSON::Schema::Modern::_get_or_load_resource at line 786 of JSON/Schema/Modern.pm, avg 35µs/call
# 6 times (1.13ms+0s) by JSON::Schema::Modern::Document::OpenAPI::_add_vocab_and_default_schemas at line 204 of JSON/Schema/Modern/Document/OpenAPI.pm, avg 188µs/call | |||||
# spent 6.28ms within Cpanel::JSON::XS::encode which was called 15 times, avg 419µs/call:
# 8 times (283µs+0s) by List::Util::first at line 174 of JSON/Schema/Modern.pm, avg 35µs/call
# 7 times (6.00ms+0s) by JSON::Schema::Modern::add_schema at line 170 of JSON/Schema/Modern.pm, avg 856µs/call | |||||
# spent 9µs within Cpanel::JSON::XS::new which was called:
# once (9µs+0s) by JSON::MaybeXS::new at line 49 of JSON/MaybeXS.pm | |||||
# spent 0s within Cpanel::JSON::XS::utf8 which was called:
# once (0s+0s) by JSON::MaybeXS::new at line 50 of JSON/MaybeXS.pm |