ç®æ¬¡ (7)
- [ããã°ç޹ä»] ããã°ã®æµå ¥ãå¢ããããã®4ãåœèªå¯Ÿå¿ â ã«ãŒãã£ã³ã°ã»ç¿»èš³ã»ç»åã»ã»ãã³ãã£ãã¯æ€çŽ¢ãšãã®éç
- 1. å€åœäººèšªåè ã«ããåã£ãèšèªãèŠããïŒURLã«ãŒãã£ã³ã°ãšèšèªæ€åº
- 2. èšäºã®ç¿»èš³ïŒGoogle Translate
- 3. ç»åã»ãµã ãã€ã«ã®ç¿»èš³ïŒç»åçæLLMã®æŽ»çš
- 4. å€èšèªæ€çŽ¢ïŒã»ãã³ãã£ãã¯ãã¯ãã«DBã§éåœèªã®åæ1ã»ãããåå©çšãã
- 5. å®éã«åãããŠã¿ãŠèŠããéçãšå¯Ÿå¿
- ãŸãšã
[ããã°ç޹ä»] ããã°ã®æµå ¥ãå¢ããããã®4ãåœèªå¯Ÿå¿ â ã«ãŒãã£ã³ã°ã»ç¿»èš³ã»ç»åã»ã»ãã³ãã£ãã¯æ€çŽ¢ãšãã®éç
ãã®èšäºã¯ AI 70% 人é 30%ã§æžããŸããã
éåœèªã ãã§éå¶ããŠããããã°ã®èšªåè æ°ãå¢ãããŠã¿ãããŠãè±èªã»æ¥æ¬èªã»äžåœèªãŸã§å«ãã4ãåœèªã«å¯Ÿå¿ããããšã«ããŸããã
å®éã®æµå ¥ã«ã€ãªããããã«ãå€åœäººèšªåè ãé©åãªèšèªãžèªåã§æ¡å ãã èšäºããã¹ãã®ç¿»èš³ â èšäºç»åã®ç¿»èš³ â å€èšèªæ€çŽ¢ãŸã§å®è£ ããŸããã
1. å€åœäººèšªåè ã«ããåã£ãèšèªãèŠããïŒURLã«ãŒãã£ã³ã°ãšèšèªæ€åº

æµ·å€ããæ€çŽ¢ããªã³ã¯çµç±ã§èšªãã人ã«éåœèªã®ç»é¢ããã®ãŸãŸèŠããããããã«é¢è±ããŠããŸããšæããŸããã
ããã§ å€åœèªã¯ /en · /ja · /zh-CN ã®ãã¬ãã£ãã¯ã¹ã§URLãåããåå蚪åæã«èšªåè
ã®èšèªã«åãããŠããç§»åããããã«ããŸãããïŒNext.jsã®proxyãæŽ»çšïŒ
åªå èšèªãžã®èªåèªå°ïŒãã¬ãã£ãã¯ã¹ã®ãªãURLã§ã¢ã¯ã»ã¹ãããšã
NEXT_LOCALEã¯ãã㌠â ãã©ãŠã¶ã®Accept-LanguageïŒqå€ã®é ãç¹äœåäžåœèªãzh-CNã察å¿èšèªããªããã°enïŒã®é ã§å€æãã該åœèšèªã®URLãž307ã§ãªãã€ã¬ã¯ãããŸããèªåã§éžãã èšèªãèšæ¶ïŒããããŒã®èšèªã¡ãã¥ãŒãèšäºäžéšã®èšèªãªã³ã¯ã§èšèªãåãæ¿ãããšã¯ãããŒã«ä¿åããã®ã§ãæ¥æ¬èªãã©ãŠã¶ã®ãŠãŒã¶ãŒãéåœèªã®åæãã¯ãªãã¯ããŠã
/jaã«æ»ãããªãããã«ããŸãããæ€çŽ¢ãããã®é€å€ïŒGooglebotããªã³ã¯ãã¬ãã¥ãŒçšã®ãããã¯ãªãã€ã¬ã¯ãã®å¯Ÿè±¡ããå€ããéåœèªã®ã€ã³ããã¯ã¹ãå®ããŸããã
2. èšäºã®ç¿»èš³ïŒGoogle Translate

翻蚳ãšã³ãžã³ã«ã¯æ¯æ50äžæåãŸã§ç¡æã® Google Cloud Translationã䜿ããŸãããå人ããã°ãªã®ã§ç¡ææ ãè¶
ããŠèª²éãããã®ã¯é¿ããããRedisã«æ¯æã®çޝèšç¿»èš³æåæ°ãèšé²ãã480,000æåãŸã§ããå®è¡ãããªãããå®å
šã©ã€ã³ãèšããŸããã
äžéãæ±ºãŸã£ãŠããã®ã§ãéä¿¡ããæåæ°ãã®ãã®ãæžããããšãéèŠã§ããããšãã£ã¿ãŒãä¿åããHTMLããã®ãŸãŸéã£ãŠã¿ããšãå®éã®æ¬æãããHTMLã¿ã°ã»ã¹ã¿ã€ã«ã»é·ãURLã»ã·ã³ã¿ãã¯ã¹ãã€ã©ã€ããããã³ãŒããããã¯ãæåæ°ã®å€§åãå ããŠããããã§ãã
<table>ãstyleãé·ãhref屿§ãŸã§ãã¹ãŠæåæ°ã«å ç®Shikiã
<span style="color:...">ã«åå²ããã³ãŒããããã¯å šäœã課é察象äžèЧãã¬ãã¥ãŒïŒ
previewïŒãæ¬æãšã¯å¥ã«éè€ããŠç¿»èš³
æ§é ã»å±æ§ã¯ããŒã«ã«ã®DOMã«ä¿æïŒçŽç²ãªããã¹ããããã¯ã¯ã¿ã°ãªãã®å¹³æïŒ
TEXTïŒã ããéãã<a>ã»<span>ã®é·ã屿§ã¯id="a0"ã«äžæçã«çœ®ãæããŠãã埩å ã€ã³ã©ã€ã³ã³ãŒãïŒ
<code>ïŒã®ä¿è·ïŒ<span class="notranslate">__CT0__</span>ã§å²ã¿ãèª€èš³ãšæåæ°ã®ç¡é§ã鲿¢ã³ãŒããããã¯ïŒ
<pre>ïŒã¯éåœèªã ããéžå¥ïŒçŽç²ãªè±èªã®ã³ãŒãã¯æåæ°0ãšããŠã¹ãããããéåœèªãæ··ãã£ãè¡ã®ã³ã¡ã³ãã»æååãªãã©ã«ã ããæœåºããŠç¿»èš³previewãããŒã«ã«ã§çæïŒç¿»èš³æžã¿ã®æ¬æHTMLããã¿ã°ã ããåãé€ããŠçæ
ãã®æ¹æ³ã§ãå®éã®èšäºããŒã¹ã§éä¿¡æåæ°ãçŽ65ã70%åæžããŸããã ã·ãªãŒãºã®ã¿ã€ãã«ã»èª¬æïŒseries_translationïŒãåããã€ãã©ã€ã³ã«ä¹ããŸããã
å®ã¯ãã®æ©èœãäœããªãããJSON圢åŒã§èšäºã®æ§é ãæŽçããŠã¬ã³ããªã³ã°ããæ¹æ³ã®ã»ããã翻蚳ããã®ä»ã®åŸåŠçã«ã¯åããŠããã®ã§ã¯ãšèããŸãããJSONãªãäœèšãªã¿ã°ãç¡çã«å¥ãããŠåŠçããå¿ èŠããªãã翻蚳ã«å¿ èŠãªæååããã£ããæãåºããããããã§ãã
3. ç»åã»ãµã ãã€ã«ã®ç¿»èš³ïŒç»åçæLLMã®æŽ»çš

èšäºã ãã翻蚳ããŠãããšãæ¬æã®å³ãã¹ã¯ãªãŒã³ã·ã§ããããµã ãã€ã«ã®äžã®éåœèªããã®ãŸãŸæ®ãåé¡ããããŸãããããã§ç»åçæã»ç·šéLLMïŒMeta Muse muse-image-1.0ïŒãã€ãªããåèšèªãžç»åãŸã§ç¿»èš³ããããã«ããŸãããç»å1æããã $ 0.01 ã®æéãããããŸãã
/images/editsAPIã§ãå ã®ã¬ã€ã¢ãŠãã»è²ã»ç¢å°ã»ã³ãŒãèå¥åã¯ãã®ãŸãŸã«ç»åå ã®éåœèªããã¹ãã ãã察象èšèªã«å·®ãæ¿ããŸããèšèªããšã®å·®ãæ¿ãå±¥æŽã¯
image_translation(source_key, lang)ã«ä¿åããŠããã翻蚳æãæåã§ä¿®æ£ããŠä¿åãããšããæ¬æã»ãµã ãã€ã«ã®ç¿»èš³ç»åãããã³ã°ããã®ãŸãŸç¶æãããããã«ããŸããã
ç»åããã€ãã©ã€ã³ã«ä¹ããããŸããããšãã©ãéåœèªãããŸã翻蚳ã§ããã«æ®ããŠããŸãããšããããŸããLLMã«ã¬ãã¥ãŒãããã°ããããã±ãŒã¹ãååž°çã«è§£æ±ºã§ããŸãããæéãšå質ã®åé¡ããããããèªåã§ç¢ºèªããªããä¿®æ£ãç¶ããæ¹åŒã«ããŸããã
4. å€èšèªæ€çŽ¢ïŒã»ãã³ãã£ãã¯ãã¯ãã«DBã§éåœèªã®åæ1ã»ãããåå©çšãã
å€èšèªæ€çŽ¢ãæ€èšãããšããæåã«æµ®ããã ã®ã¯ããã§ããã
ããã¯ãã«DBã¯ã©ããã»ãã³ãã£ãã¯ïŒæå³ããŒã¹ïŒãªDBã ãããèšèªãéã£ãŠãæå³ãè¿ããã°è¿ããã¯ãã«äœçœ®ã«æ¥ãã®ã§ã¯ïŒã
èšèªããšã«ãã¯ãã«DBã4åã«è€è£œãã代ããã«ãå€èšèªã¯ãã¹åã蟌ã¿ã«å¯Ÿå¿ããWeaviateã®ã¢ãã«ïŒSnowflake/snowflake-arctic-embed-l-v2.0ïŒãä¿¡ããŠãéåœèªã®åæ1ã»ããã ããã€ã³ããã¯ã¹ããç¶æ
ã§ã4èšèªã®æ€çŽ¢èªããã®ãŸãŸãã¯ãã«æ€çŽ¢ããããã«ããŸããã
flowchart TB
Req["æ€çŽ¢ãªã¯ãšã¹ã (?q=&lang=)"] --> Norm["Unicodeæ£èŠå"]
Norm --> Branch{"ãªã¯ãšã¹ãèšèª"}
Branch -->|"ko"| KoVec["Weaviate ãã€ããªãã (α = 0.75)"]
Branch -->|"en / ja / zh-CN"| EnVec["Weaviate çŽç²ãã¯ãã« (α = 1.0)"]
KoVec --> KoDB["éåœèªã®å
¬éèšäºãååŸ"]
EnVec --> EnDB["å
¬é翻蚳çã®ãã£ã«ã¿ · ããŒãžã³ã°"]
KoDB -.->|"倱æ · 0ä»¶ãã©ãŒã«ããã¯"| PgKo["PostgreSQL pg_trgm"]
EnDB -.->|"倱æ · 0ä»¶ãã©ãŒã«ããã¯"| PgEn["翻蚳ç pg_trgm"]4èšèªçµ±åã®èªç¶èšèªæ€çŽ¢ãããŒ
Unicodeæ€çŽ¢èªã®ãµãã¿ã€ãºïŒæ¢åã®æ£èŠè¡šçŸãéåœèªã»è±åããæ®ããªãã£ãããã§ãæ¥æ¬èªã»äžåœèªã®æ€çŽ¢èªãäžžããšæ¶ããŠããåé¡ããUnicodeã®æåã»æ°åããŒã¹ã«çŽããŸããã
éåœèªãšå€åœèªã®éã¿ãåé¢ïŒéåœèªã¯ãã¯ãã« + BM25ã®ãã€ããªããïŒ
alpha = 0.75ïŒã§æ¢ããå€åœèªã§ã¯éåœèªã®BM25ãããã£ãŠéªéã«ãªãããçŽç²ãã¯ãã«ïŒalpha = 1.0ïŒã§åè£èšäºã®IDãæœåºãããã®èšèªã®å ¬é翻蚳çã ããçµã蟌ãã§è¿ããŸããçµæããªãå Žåããã¯ãã«æ€çŽ¢ã倱æããå Žåã¯ãPostgreSQLã®pg_trgmããŒã¯ãŒãæ€çŽ¢ã«ãã©ãŒã«ããã¯ããŸãã
5. å®éã«åãããŠã¿ãŠèŠããéçãšå¯Ÿå¿
ãããŸã§çµã¿èŸŒãã§å®éã®èšäºã翻蚳ããŠã¿ããšãèªååã ãã§ã¯å®ç§ã«ãªããªã2ã€ã®éçãã¯ã£ããèŠããŸããã
ã³ãŒãã®æ§æãå£ããªããããã³ãŒããããã¯å ã§ã¯ã³ã¡ã³ãã»æååã®ãã¿ãŒã³ã ããéžãã§ç¿»èš³ããŠããããããã¿ãŒã³ã®å¢çã«åŒã£ããã£ãäžéšã®ã³ãŒããããã¯ã®ã³ã¡ã³ãã®éåœèªã翻蚳ãããã«ãã®ãŸãŸæ®ãã±ãŒã¹ããšãã©ãçºçããŸããã
ç»åçæLLMãå³ã®äžã®å°ããªéåœèªãå€ããããªãã£ãããæåã厩ãããããããšããããŸãã倿ŽåŸã®ç»åãå¥ã®LLMã«ã¬ãã¥ãŒãããŠå詊è¡ããã°è§£æ±ºã§ããããšã¯ç¢ºèªããŸããããããã§ãã¬ã€ã¢ãŠãããã©ã³ãã®å質ãäžèªç¶ã«ãªãå ŽåããããŸãã
ãã®2ã€ã®åé¡ããããããæåã§ãä¿®æ£ããŠç¿»èš³æãšç¿»èš³ç»åãå·®ãæ¿ããããããã«ããŸãããäºç®ã«äœè£ãããã°ãLLMã«ããã¬ãã¥ãŒãŸã§è¡ã£ãŠå翻蚳ãããå質ãäžãããããšæããŸãã
ãŸãšã
ã³ãŒããããã¯ã®ã³ã¡ã³ãæŒããç»å翻蚳ã®å質ã®ããã«ããŸã æãå ¥ããã¹ããšããã¯æ®ã£ãŠããŸãããæã®ç¡ææ ã®ç¯å²ã§4ãåœèªã®ããã°ãç¶ç¶çã«éå¶ããæµ·å€ããã®æµå ¥ã詊ããåºç€ã¯ååã«æŽããŸããã
ä»åã®æ©èœè¿œå ã§ãããã°ã®èªè ããã£ãšå¢ããŠããããå¬ããã§ãã
