Hyphenopoly.js

Hyphenation for node and Polyfill for client-side hyphenation.

String

mnater/Hyphenopoly mnater.github.io/Hyphenopoly/hyphenopoly

README

Hyphenopoly.js

Hyphenopoly.js is a __JavaScript-polyfill for hyphenation in HTML__: it hyphenates text if the user agent does not support CSS-hyphenation at all or not for the required languages and it is a __Node.js-module__.

The package consists of the following parts:

- _Hyphenopoly_Loader.js_ (11KB unpacked, 2KB minified and compressed): feature-checks the client and loads other resources if necessary.

- _Hyphenopoly.js_ (36KB unpacked, 5KB minified and compressed): does the whole DOM-foo and wraps wasm.

- _wasm-Modules_ (sizes differ! e.g. en-us.wasm: 21KB uncompressed, 15KB compressed): core hyphenation functions and hyphenation patterns in a space saving binary format (including pattern license).

- _hyphenopoly.module.js_: the node module to hyphenate plain text strings.

Usage (Browser)

Place all the code for Hyphenopoly at the top of the header (immediately after the `` tag) to ensure resources are loaded as early as possible.<div class="base"> </div><div class="base">You'll have to insert two script blocks. In the first block, load Hyphenopoly_Loader.js as an external script.</div><div class="base">In the second block, provide the initial configurations for Hyphenopoly_Loader as an inline script. This also triggers all further steps.</div><div class="base"> </div><div class="base">Also, don't forget to enable CSS hyphenation.</div><div class="base"> </div><div class="base"><a href="http://mnater.github.io/Hyphenopoly/min/example.html" title="Example" target="_blank" rel="nofollow noopener noreferrer">Example</a>:</div><div class="code"><ol start="0"><li>```html</li><li><!DOCTYPE html></li><li><html></li><li> <head></li><li> <meta http-equiv="content-type" content="text/html; charset=UTF-8"></li><li> <title>Example 1</title></li><li> <script src="./Hyphenopoly_Loader.js"></script></li><li> <script></li><li> Hyphenopoly.config({</li><li> require: {</li><li> "la": "honorificabilitudinitas",</li><li> "de": "Silbentrennungsalgorithmus",</li><li> "en-us": "Supercalifragilisticexpialidocious"</li><li> },</li><li> setup: {</li><li> selectors: {</li><li> ".container": {}</li><li> }</li><li> }</li><li> });</li><li> </script></li><li> <style type="text/css"></li><li> body {</li><li> width:60%;</li><li> margin-left:20%;</li><li> }</li><li> p {</li><li> text-align: justify;</li><li> margin: 0 2em 0 0;</li><li> }</li><li> .container {</li><li> display: flex;</li><li> hyphens: auto;</li><li> -ms-hyphens: auto;</li><li> -moz-hyphens: auto;</li><li> -webkit-hyphens: auto;</li><li> }</li><li> </style></li><li> </head></li><li> <body></li><li> <h1>Example 1</h1></li><li> <div class="container"></li><li> <p lang="la">Qua de causa Helvetii quoque reliquos Gallos virtute praecedunt, quod fere cotidianis proeliis cum Germanis contendunt, cum aut suis finibus eos prohibent aut ipsi in eorum finibus bellum gerunt.</p></li><li> <p lang="en-us">For which reason the Helvetii also surpass the rest of the Gauls in valor, as they contend with the Germans in almost daily battles, when they either repel them from their own territories, or themselves wage war on their frontiers.</p></li><li> <p lang="de">Aus diesem Grund übertreffen auch die Helvetier die übrigen Gallier an Tapferkeit, weil sie sich in fast täglichen Gefechten mit den Germanen messen, wobei sie diese entweder von ihrem Gebiet fernhalten oder selbst in deren Gebiet kämpfen.</p></li><li> </div></li><li> </body></li><li></html></li><li>```</li></ol></div><div class="base">Let's go through this example step by step:</div><div class="base"> </div><div class="base"><h3>UTF-8</h3></div><div class="base">Make sure your page is encoded as utf-8.</div><div class="base"> </div><div class="base"><h3>script blocks – load, configure and run Hyphenopoly_Loader.js</h3></div><div class="base">Hyphenopoly_Loader.js needs some information to run. This information is provided as a parameter object to the function Hyphenopoly.config(). This information is stored in a globally accessible Object called window.Hyphenopoly. Hyphenopoly_Loader.js and (if necessary) Hyphenopoly.js will add other methods and properties only to this object – there will be no other global variables or functions beyond this object.</div><div class="base"> </div><div class="base"><h4>require</h4></div><div class="base">The configuration object must have exactly one property called require which itself is an object containing at least one nameValuePair where the name is a language code string (Some languages are region-specific. See the patterns directory for supported languages. E.g. just using en won't work, use en-usor en-gb) and the value is a long word string in that language (preferably more than 12 characters long).</div><div class="base"> </div><div class="base">If you want to force the usage of Hyphenopoly.js for a language (e.g. for testing purposes), write "FORCEHYPHENOPOLY" instead of the long word.</div><div class="base"> </div><div class="base">Hyphenopoly_Loader.js tests if the client (aka browser, aka user agent) supports CSS hyphenation for the language(s) given in require. In the example above, it will test if the client supports CSS-hyphenation for Latin, German and US-English.</div><div class="base"> </div><div class="base">If one of the given languages isn't supported, it automatically hides the document's contents and loads Hyphenopoly.js and the necessary WebAssembly modules.</div><div class="base"> </div><div class="base">Hyphenopoly.js – once loaded – will hyphenate the elements according to the settings and unhide the document when it's done.</div><div class="base"> </div><div class="base">If something goes wrong and Hyphenopoly.js is unable to unhide the document, Hyphenopoly_Loader.js has a timeout that kicks in after some time (defaults to 1000ms) and unhides the document and writes a message to the console.</div><div class="base"> </div><div class="base">If the browser supports all required languages, the script deletes the Hyphenopoly-object and terminates without further ado.</div><div class="base"> </div><div class="base"><h3>enable CSS-hyphenation</h3></div><div class="base">Hyphenopoly by default hyphenates elements (and their children) with the classname .hyphenate. Don't forget to enable CSS-hyphenation for the classes eventually handled by Hyphenopoly.</div><div class="base"> </div><div class="base"><h2>Usage (node)</h2></div><div class="base"><a href="https://npm.runkit.com/hyphenopoly" title="Try hyphenopoly on RunKit" target="_blank" rel="nofollow noopener noreferrer"><img src="https://badge.runkitcdn.com/hyphenopoly.svg" alt="Try hyphenopoly on RunKit"/></a></div><div class="base"> </div><div class="base">Install:</div><div class="code"><ol start="0"><li>````shell</li><li>npm i hyphenopoly</li><li>````</li></ol></div><div class="base"> </div><div class="code"><ol start="0"><li>````javascript</li><li>import hyphenopoly from "hyphenopoly";</li><li> </li><li>const hyphenator = hyphenopoly.config({</li><li> "require": ["de", "en-us"],</li><li> "hyphen": "•",</li><li> "loader": async (file) => {</li><li> const {readFile} = await import("node:fs/promises");</li><li> const {dirname} = await import("node:path");</li><li> const {fileURLToPath} = await import("node:url");</li><li> const cwd = dirname(fileURLToPath(import.meta.url));</li><li> return readFile(`${cwd}/../patterns/${file}`);</li><li> },</li><li> "exceptions": {</li><li> "en-us": "en-han-ces"</li><li> }</li><li>});</li><li> </li><li>async function hyphenate_en(text) {</li><li> const hyphenateText = await hyphenator.get("en-us");</li><li> console.log(hyphenateText(text));</li><li>}</li><li> </li><li>async function hyphenate_de(text) {</li><li> const hyphenateText = await hyphenator.get("de");</li><li> console.log(hyphenateText(text));</li><li>}</li><li> </li><li>hyphenate_en("hyphenation enhances justification.");</li><li>hyphenate_de("Silbentrennung verbessert den Blocksatz.");</li><li>````</li></ol></div><div class="base"> </div><div class="base"><h2>Support this project</h2></div><div class="base"><a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=SYNZKB8Z2FRQY" title="PayPal" target="_blank" rel="nofollow noopener noreferrer"><img src="https://www.paypal.com/en_US/i/btn/btn_donateCC_LG_global.gif" alt="PayPal"/></a></div><div class="base"> </div><a href="https://opencollective.com/hyphenopoly/donate" target="_blank" rel="nofollow noopener noreferrer" target="_blank"> <img src="https://opencollective.com/hyphenopoly/donate/button@2x.png?color=blue" width=300 /></a><div class="base"> </div><div class="base"><h2>Automatic hyphenation</h2></div><div class="base">The algorithm used for hyphenation was developed by Franklin M. Liang for TeX. It works more or less like this:</div><div class="base"> </div><div class="base">1. Load a set of precomputed language specific patterns. The patterns are stored in a structure called a trie, which is very efficient for this task.</div><div class="base">2. Collect all patterns that are a substring of the word to be hyphenated.</div><div class="base">3. Combine the numerical values between characters: higher values overwrite lower values.</div><div class="base">4. Odd values are hyphenation points (except if the hyphenation point is left from leftmin and right from rightmin), replace them with a soft hyphen and drop the other values.</div><div class="base">5. Repeat steps 2. - 4. for all words longer than minWordLength</div><div class="base"> </div><div class="base">Example:</div><div class="code"><ol start="0"><li>````text</li><li>Hyphenation</li><li>h y p h e n a t i o n</li><li>h y3p h</li><li> h e2n</li><li> h e n a4</li><li> h e n5a t</li><li> 1n a</li><li> n2a t</li><li> 1t i o</li><li> 2i o</li><li> o2n</li><li>h0y3p0h0e2n5a4t2i0o2n</li><li>Hy-phen-ation</li><li>````</li></ol></div><div class="base"> </div><div class="base">The patterns are precomputed and available for many languages on <a href="https://www.ctan.org/tex-archive/language/hyphenation/" title="CTAN" target="_blank" rel="nofollow noopener noreferrer">CTAN</a> and <a href="https://github.com/hyphenation/tex-hyphen" title="tex-hyphen" target="_blank" rel="nofollow noopener noreferrer">tex-hyphen</a>. For Hyphenopoly.js they are converted to a succinct trie data structure (including pattern license, metadata, and the patterns).</div><div class="base"> </div><div class="base">The original patterns are computed from a large list of hyphenated words by a program called patgen. They aim to find some hyphenation points – not all – because it's better to miss a hyphenation point than to have some false hyphenation points. Most patterns are really good, but none are error free.</div><div class="base"> </div><div class="base">These patterns vary in size. This is mostly due to the different linguistic characteristics of the languages.</div><div class="base"> </div><div class="base"><h2>Contributors ✨</h2></div><div class="base"> </div><div class="base">Thanks goes to these wonderful people (<a href="https://allcontributors.org/docs/en/emoji-key" title="emoji key" target="_blank" rel="nofollow noopener noreferrer">emoji key</a>):</div><div class="base"> </div><table> <tbody> <tr> <td align="center"><a href="https://github.com/StephanHoyer" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars.githubusercontent.com/u/54701?v=4?s=100" width="100px;" alt="Stephan Hoyer"/> Stephan Hoyer</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=StephanHoyer" target="_blank" rel="nofollow noopener noreferrer" title="Documentation">📖</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=StephanHoyer" target="_blank" rel="nofollow noopener noreferrer" title="Code">💻</a></td> <td align="center"><a href="http://thomasbroadley.com" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars0.githubusercontent.com/u/8731922?v=4?s=100" width="100px;" alt="Thomas Broadley"/> Thomas Broadley</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=tbroadley" target="_blank" rel="nofollow noopener noreferrer" title="Documentation">📖</a></td> <td align="center"><a href="https://kailueke.gitlab.io/" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars0.githubusercontent.com/u/1189130?v=4?s=100" width="100px;" alt="Kai Lüke"/> Kai Lüke</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=pothos" target="_blank" rel="nofollow noopener noreferrer" title="Code">💻</a></td> <td align="center"><a href="http://www.data-factory.net/" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars2.githubusercontent.com/u/998558?v=4?s=100" width="100px;" alt="Sebastian Blank"/> Sebastian Blank</a> <a href="#example-blankse" title="Examples">💡</a></td> <td align="center"><a href="https://www.ghsvs.de" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars2.githubusercontent.com/u/20780646?v=4?s=100" width="100px;" alt="ReLater"/> ReLater</a> <a href="#maintenance-ReLater" title="Maintenance">🚧</a></td> <td align="center"><a href="https://github.com/julian-zatloukal" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars3.githubusercontent.com/u/58230917?v=4?s=100" width="100px;" alt="julian-zatloukal"/> julian-zatloukal</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=julian-zatloukal" target="_blank" rel="nofollow noopener noreferrer" title="Documentation">📖</a></td> <td align="center"><a href="http://www.neoskop.de/" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars1.githubusercontent.com/u/1702250?v=4?s=100" width="100px;" alt="Maik Jablonski"/> Maik Jablonski</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=jablonski" target="_blank" rel="nofollow noopener noreferrer" title="Documentation">📖</a></td> </tr> <tr> <td align="center"><a href="https://github.com/yashha" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars0.githubusercontent.com/u/4728786?v=4?s=100" width="100px;" alt="yashha"/> yashha</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=yashha" target="_blank" rel="nofollow noopener noreferrer" title="Code">💻</a></td> <td align="center"><a href="http://danburzo.ro/" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars3.githubusercontent.com/u/205375?v=4?s=100" width="100px;" alt="Dan Burzo"/> Dan Burzo</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=danburzo" target="_blank" rel="nofollow noopener noreferrer" title="Code">💻</a></td> <td align="center"><a href="https://github.com/CommanderRoot" target="_blank" rel="nofollow noopener noreferrer"><img src="https://avatars.githubusercontent.com/u/4395417?v=4?s=100" width="100px;" alt="Tobias Speicher"/> Tobias Speicher</a> <a href="https://github.com/mnater/Hyphenopoly/commits?author=CommanderRoot" target="_blank" rel="nofollow noopener noreferrer" title="Code">💻</a></td> </tr> </tbody> <tfoot><div class="base"> </div> </tfoot></table><div class="base"> </div><div class="base"> </div><div class="base"> </div><div class="base">This project follows the <a href="https://github.com/all-contributors/all-contributors" title="all-contributors" target="_blank" rel="nofollow noopener noreferrer">all-contributors</a> specification. Contributions of any kind welcome!</div><div class="base"> </div></div></div></div><div id="footer"><div class="footer-box"><div class="footer-info"><div class="fi-logo"><img src="/img/footer-logo.png" width="65"/></div><div class="fi-intro">为什么是最好的 GitHub 项目？技术的发展速度比以往任何时候都快，技术正在全速创新。几乎每天都会发布惊人的开源项目。 如何了解最新趋势？如何快速检查真正重要的项目，现在而不是 6 个月前？探客时代每天拍摄 GitHub stars 的 “快照”，以获得 2000 多个项目的精选列表，以检测过去几个月的趋势。 你想要更多的项目吗？我们不会扫描 GitHub 上的所有现有项目，而是根据我们的经验和我们在 Internet 上阅读的内容，专注于我们发现 “有趣” 的精选项目列表。想要了解的项目可以提交给我们。</div><div class="fi-qrcode"><h3>关注公众号</h3><img src="/img/qrcode.jpg" width="272" height="272"/></div></div><div class="footer-copyright">© 2022, 北京探客时代网络科技有限公司　 <a href="http://beian.miit.gov.cn" target="_blank" rel="nofollow noopener noreferrer">京 ICP 备 2022008592 号</a>　<a href="http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11011402012574" target="_blank" rel="nofollow noopener noreferrer">京公网安备 11011402012574 号</a>　<a href="contact.html" target="_blank" rel="nofollow noopener noreferrer">信息举报</a></div></div></div><script>function request(t,e,n){$.ajax({url:t,type:"get",data:e,dataType:"json",success:function(t){n(t.data)}})}function levenshtein(t,e){t=t.toLowerCase(),e=e.toLowerCase();for(var n=t.length,o=e.length,i=[],a=0;a<n+1;a++)i.push(new Array(o+1));for(a=0;a<=n;a++)i[a][0]=a;for(a=0;a<=o;a++)i[0][a]=a;var l,s=t.split(""),r=e.split("");for(a=1;a<=n;a++)for(var c=1;c<=o;c++){l=s[a-1]==r[c-1]?0:1;var u=i[a-1][c-1]+l,h=i[a][c-1]+1,f=i[a-1][c]+1;i[a][c]=Math.min(u,h,f)}return 1-i[n][o]/Math.max(t.length,e.length)}$(function(){var t=[],e=[];request("https://api.tkcnn.com/gh/tag/all",null,function(e){t=e;var n=window.location.href.match(/([\w-]+)\.html/);if(n){var o=t.find(t=>t.code===n[1]);o&&$(".select-set").val(o.name)}}),$(".select-set").on("click input",function(){var n=$(this).val();if(n=n.replace(/(^\s+|\s+$)/g,""),e=[],""===n.replace(/s/g,""))e=t;else{for(var o,i=0;i<t.length;i++)(o=levenshtein(n,t[i].name))>=.1&&e.push({name:t[i].name,code:t[i].code,count:t[i].count,similar:o});e.sort(function(t,e){return e.similar-t.similar})}var a="";for(i=0;i<e.length;i++)a+=`<button>${e[i].name} (${e[i].count})</button>`;$(".select-list").html(a)}),$(".select-list").on("click","button",function(t){$(".select-list").hide(),setTimeout(function(){$(".select-list").css("display","")},200),window.location.href=window.location.origin+`/repo/${e[$(this).index()].code}.html`})});</script></body></html>