ç®æ¬¡ (15)
- æ0ãŠã©ã³ã§ããã°èªç¶èšèªæ€çŽ¢æ©èœã远å ãã
- 奜ããªèšäºãèŠã€ããã®ã¯é£ããããŒã¯ãŒããæ€çŽ¢
- ããŸã蚪ããªãããã°ããµãŒããŒã«ãã£ãšãéããããããªãïŒ
- ãã€ã§ãæ¶ãããšãããã€ã³ãã§
- å šäœæ§é
- æçš¿ã玢åŒä»ãããæ¹æ³
- æãå€ãããšã€ã³ããã¯ã¹ãå€ãããŸãã
- ã©ã®èšäºãã€ã³ããã¯ã¹ã«ç»é²ãããããèšé²ããŸãã
- æ€çŽ¢ããæ¹æ³
- æ€çŽ¢ãããŒ
- ããã³ãã¯ã»ãšãã©å€ãããŸããã§ããã
- é害ããã£ãŠãæ€çŽ¢ã¯ã§ããŸãã
- Weaviateã䜿ããªãå Žåã¯ããŒã¯ãŒãæ€çŽ¢ã«ä»£ãã
- äœä»æ§ãµãŒããŒ/ç¡æå¶çŽãå æããããã®èœå
- çµãã
æ0ãŠã©ã³ã§ããã°èªç¶èšèªæ€çŽ¢æ©èœã远å ãã
ããã°æ€çŽ¢ããã¯ã¹ã«ãmdãšhtmlã®éãããšæ€çŽ¢ãããšãã [AI Coding] mdãšhtml圢åŒã«ã€ããŠã®èããæãæ€çŽ¢ãããããããšæããŸããããšããã§ãäžèšã®ããã«æ€çŽ¢ãããšãããã°ã«äœã®èšäºãåºãŠããŸããã§ããã
ãã®èšäºã§ã¯ãããŒã¯ãŒãæ€çŽ¢ã ãã§ãã£ãå人ããã°ã«æç« ã®æå³ã§æãæ¢ãèªç¶èšèªæ€çŽ¢ãä»ããéçšããŸãšããŸããã
ç®æšã¯2ã€ãããŸãã
2 vCPU / 4GBã®RAMã§ããããã¯ã¿ãŒã®åã蟌ã¿ãšãã¯ã¿ãŒDBã®æ€çŽ¢ãå¯èœã§ãªããã°ãªããŸããã
远å éã䜿ããªãã§ãã ããã
奜ããªèšäºãèŠã€ããã®ã¯é£ããããŒã¯ãŒããæ€çŽ¢
å ã®æ€çŽ¢ã¯MySQL FULLTEXTããŒã¯ãŒãæ€çŽ¢ã§ãããæ£ç¢ºãªåèªãç¥ãã°ããæ¢ããŸãããèªè ã¯æ®éã®åèªã§ã¯ãªããèŠããã§ããåé¡ã§æ€çŽ¢ããŸãã
ããã°ã€ã³ãã©ãã©ãè§£æŸãããŸãã â æçš¿æ: JWTãªãã¬ãã·ã¥ããŒã¯ã³ããŒããŒã·ã§ã³
ãä¿åããŸãããç»é¢ãå€ãããªãã â æ¢ãæçš¿: Next.js ãã£ãã·ã¥ç¡å¹å
ã ããæã®æå³ãè¿ãæãæ¢ããŠãããæ€çŽ¢ãå¿ èŠã§ããã
ããŸã蚪ããªãããã°ããµãŒããŒã«ãã£ãšãéããããããªãïŒ
ãµãŒããŒã¯ãMySQLãRedisãSpring ServerãJenkinsãNginxããã¹ãŠäžã€ã«æµ®ããã§ããŸããã ããã远å ã®ãã¯ãã«DBã远å ããã«ã¯è² æ ã倧ããã£ããçä¿¡èŠæ±åŠçã«å¿ èŠãªäœè£Ramã確ä¿ããŠãããªããã°ãªããªãããã§ãã ïŒç¹ã«Jenkinsã§ãã«ãäžã¯500MBãæ®ããŸããïŒ
ããã§ããã¯ãã«DBãšåã蟌ã¿ã¯ç¡æã§ãã¯ãã«DBãšåã蟌ã¿ãæäŸããã¯ã©ãŠããµãŒãã¹ããããæ¢ããŠã¿ãŸãããã ããèŠã€ããã®ãWeaviate Cloudç¡æãã©ã³ã§ãã
1GB RAM / 10GB SSD / 10äžåã®ãã¯ãã«ãªããžã§ã¯ããç¡æã§ä¿åã§ããŸãããããŠ1æ¥2000åç¡æåã蟌ã¿ãšæ1000åRAGæ€çŽ¢ãå¯èœã§ãã
äžæ¥ã®èšªåè ããæåæ°ãèæ ®ããæãç¡æãã©ã³ã§ãååå¯èœã ãšå€æããŸããã
ãã€ã§ãæ¶ãããšãããã€ã³ãã§
WeaviateããªãŒãµãŒããŒã¯ç¡æã ã£ãã®ã§ããã€ã§ããµãŒããŒã«é害ãçºçãããã飲ã¿èŸŒãå¯èœæ§ããããšæããŸãããã ããFallbackã§æ¢åã®MySQLããŒã¯ãŒãæ€çŽ¢ãšããŠæ©èœã§ããããã«ããŸããã
èšäºã®ãœãŒã¹ã¯åžžã«MySQLã§ãã
ãã¯ãã«ã¹ãã¢ã空ãŸãã¯åæ¢ããŠãæ€çŽ¢ã¯å€±æããªãã§ãã ããããã®æã¯ããŒã¯ãŒãæ€çŽ¢ã«ä»£ãã£ãŠããŸãã
ã€ã³ããã¯ã¹ãæ¶ããŠã管çè APIãäžåºŠã«åäœæã§ããŸãã
å šäœæ§é

Weaviateã«ã¯ãæç« ã®ã¿ã€ãã«ãæ¬æãåå²ããéšåãã«ããŽãªãŒãæç« çªå·ãäžç·ã«ä¿åããŸããã¿ã€ãã«ãšæ¬æã¯æå³ãæ¯èŒããææã§ãã«ããŽãªã¯ãã£ã«ã¿ã§æžããŠããŸããããããæ€çŽ¢çµæã§åãåºãã®ã¯ããã®æãšæå³ãè¿ãæãäœåæããã ãã§ããç»é¢ã«è¡šç€ºããã¿ã€ãã«ããµã ãã€ã«ãªã©ã®ã«ãŒãæ å ±ã¯Weaviateããã€ã³ããŒããããŸããã
MySQLã¯å®éã«èŠããæãè¿ããŸãããããŠå ¬éå¯åŠãMySQLæ å ±ãããšã«ç¢ºèªããŠãã£ã«ã¿ãªã³ã°ããŸãã

ããã«ããããã¯ãã«ã¹ãã¢ãå°ãé ããŠããŠãããã©ã€ããŒããªæçš¿ãåé€ããæçš¿ãæ€çŽ¢ã«åºãªãããã«ãªããŸããã«ãŒãã«ã¯åžžã«MySQLã®ææ°ã®å 容ãèŠããŸãã
æçš¿ã玢åŒä»ãããæ¹æ³
äžæãäžžããšåã蟌ããšé·ãæã§ããã»ã©äžéå 容ãåããããŸããããã§æã圫å»ïŒãã£ã³ã¯ïŒã«åããŠå ¥ããŸãããã¡ãããäœäººã«åãããšãã«æè¯ã®çµæãåºãŠããã®ããè©äŸ¡ããããã®è©äŸ¡åºæºãèšããŠãã©ããããè¯ããªã£ãããã§ãã¯ããã°ããã®ã§ãããå人ã®ããã°ã«ãã®çšåºŠãã¹ãããã«ã¯é床ã«ããŠããŸããã§ããã
HTMLã§ã¯ãªãå¹³æãæžããŸãããšãã£ã¿ãä¿åãããšãã«ããŒã¯ããŠã³èšå·ãåŒããããã¹ãã§ãã
1,000æååäœã§æ©æ¢°çã«åã£ãŠååŸçã100æåãã€éããŸããå¢çã§åãåãããæç« ãççã«ã¯ååŸã®æèãæ®ãããã«ããŸãã
ã¿ã€ãã«ãäžç·ã«å ¥ããŸããã¿ã€ãã«ã ãã«ããåèªã§ãèŠã€ããããšãã§ããã¯ãã§ãããã
äœåããšã« article:{æçªå·}:{ åçªå·} ã§äœã£ãUUIDãäžããŸãããããŠæä¿®æ£ãªã©ãããæã¯article IDã§ç»è«ããããã¹ãŠã®éšåãæ¶ããŠåç»é²ããŸãã

Weaviateã«ã¯äœåããšã«ãã®æ å ±ã®ã¿ãä¿åããŸãã
ãããã㣠| çšé |
|---|---|
articleId, chunkIndex | ã©ã®æç« ã®äžéšããåé€æã«æ¢ã |
category | ã«ããŽãªã«åãã |
title, content | ãã¯ãã«ã«çœ®ãæãã + trigram ããŒã¯ãŒãæ€çŽ¢ |
æãå€ãããšã€ã³ããã¯ã¹ãå€ãããŸãã

æçš¿ã¯ä¿®æ£/åé€ããããšãæ¢åã®ãã¯ãã«ã€ã³ããã¯ã¹ãåé€ããŠåäœæããŸãããããŠä»¥äžã®ããã«åäœããŸãã
1. èšäºã®ä¿åã¯çµ¶å¯Ÿã«ãããã¯ããŸãããã€ã³ããã¯ã¹ã¯ä¿åïŒã³ãããïŒãçµãã£ãåŸã«å¥ã«æ»ããŸãã Weaviateãé ããæ¢ãŸã£ãŠããŠãæã¯ä¿åãããŸããå§ããæãããWeaviateã®ãããæç« ã®ä¿åã倱æããå Žåã¯ééã£ãèšèšããšæ±ºããŸããã
2. ã€ã³ããã¯ã¹ã¯ã¹ã¬ããã²ãšã€ã«ã ãè¿ããŸããå°ããªãµãŒããŒãªã®ã§ãã€ã³ããã¯ã¹ã®ããèªè ãªã¯ãšã¹ããåŠçãããªãœãŒã¹ãæžã£ãŠã¯ãããŸããã
3. åŠçæã«DBãå床èªã¿èŸŒã¿ãŸããã€ãã³ãã«ã¯æçªå·ã ããå ¥ããå®éã®ç¶æ ã¯MySQLã§å確èªããŸãã (ã€ã³ããã¯ã¹ãã©ããMySQLã確èª)
4. 倱æããŠãå詊è¡ããŸããããã°ã ããæ®ããååŸ©ã¯æååã€ã³ããã¯ã¹ã«ä»»ããŸãã
ã©ã®èšäºãã€ã³ããã¯ã¹ã«ç»é²ãããããèšé²ããŸãã
æããšã«last_embedded_atåã眮ããŠããæžããŠããŸãã
å€ | æå³ |
|---|---|
èŠèŠ | ãã®æç¹ã§ä»ã®å 容ã§çŽ¢åŒä»ããããŠããŸãã |
NULL | ãŸã ã€ã³ããã¯ã¹ã«ãªã£ãŠããªãããããäžåºŠãããŸãã |
ã€ã³ããã¯ã¹ãå¿ èŠãªç¬éïŒä¿®æ£ãçºè¡ã®åãæ¿ãïŒã«last_embedded_atã®å€ãNULLã§ç©ºã«ããŸãããããŠãããããšã«ã€ã³ããã¯ã¹ãå¿ èŠãã©ããã倿ããåºæºãšããŠäœ¿ããŸãã
æ€çŽ¢ããæ¹æ³
æ€çŽ¢ãããŒ

æ€çŽ¢èªã確èªããŠãã ããã 2æå以äžãæå€§100æåã§ãã
åãã¯ãšãªïŒãšã«ããŽãªïŒã§äœæãããã©ã³ãã³ã°ã®ãªã¹ãããã£ãã·ã¥ã«ãããã©ãããããã°åã蟌ã¿ã䜿ããã«ããã«4çªã«è¡ããŸãã
ããã§ãªããã°ã仿¥äœ¿çšã§ããéãæ®ã£ãŠããããšã確èªããŠWeaviateã§æ€çŽ¢ããŸãïŒãã¯ãã«75ïŒ +ããŒã¯ãŒã25ïŒ ãåè£100åã2ç§å¶éïŒããã©ã°ã¡ã³ãåäœã®çµæãæåäœã§ãŸãšããŠæå€§50ã®ã©ã³ãã³ã°ãªã¹ããäœæãã10åéãã£ãã·ã¥ããŸãã
ã«ãŒãœã«äœçœ®ããå¿ èŠãªã ãæ°åãåãåããMySQLããååŸããŸããå ¬éæ¡ä»¶ãå確èªããé äœã¯ãã®ãŸãŸç¶æããŸãã
æ¢åã®ããŒã¯ãŒãæ€çŽ¢ãšãŸã£ããåã圢åŒã§å¿çããŸãã
ããŒãžããŒã·ã§ã³ãç¡éã¹ã¯ããŒã«ã«ãªãã«ã¯ãæ¢åã®æ€çŽ¢èªããã£ãã·ã¥ããã®ãè¯ããšæããŸãããããã§ãªããã°åã蟌ã¿ç¶ããŸããããã ãããããªæµããåºãŸããã
ããã³ãã¯ã»ãšãã©å€ãããŸããã§ããã
ã¬ã¹ãã³ã¹åœ¢åŒãæ¢åã®ããŒã¯ãŒãæ€çŽ¢ãšåãããã«åãããŸãããã ããæ€çŽ¢çµæç»é¢ã¯ãã®ãŸãŸã§ããããã³ãã§å€ããã®ã¯3ã€ãããŸãã
管çè çšã®ã€ã³ããã¯ã¹ç®¡çããŒãžãäœæããŸãããæããšã«ã€ã³ããã¯ã¹ãã©ãããèŠãŠåã€ã³ããã¯ã¹ãå®è¡ã§ããŸãã
æ€çŽ¢ããã¯ã¹ã®ãã¬ãŒãºããèªç¶èšèªæ€çŽ¢ïŒæå€§100æåïŒãã«å€æŽããæ€çŽ¢çµæã«ããã¯ã¿ãŒ+ããŒã¯ãŒããã€ããªãããæ¡å ãå ¥ããŸããã
æ€çŽ¢èªã100æåã«å¶éããŸããã
é害ããã£ãŠãæ€çŽ¢ã¯ã§ããŸãã
ç¡æã®ã¯ã©ãŠãã¯é ããªããããããªããæ¢ãŸããããããªãããæ¶ãããããããŸãããã ããã©ããªå Žåã§ããæ€çŽ¢APIã¯ãµãŒããŒãšã©ãŒïŒ5xxïŒãåºããªãããã«ããŸããã
Weaviateã䜿ããªãå Žåã¯ããŒã¯ãŒãæ€çŽ¢ã«ä»£ãã
ãªã¯ãšã¹ã | Weaviate ã¹ããŒã¿ã¹ | çµæ |
|---|---|---|
æåã®ããŒãž | 倱æã2ç§è¶ ã仿¥äœ¿çšéè¶ é | ããŒã¯ãŒãæ€çŽ¢ã®æåã®ããŒãžã衚瀺ããŸãã |
æåã®ããŒãž | æ£åžžã§ãããçµæ0ä»¶ | ããŒã¯ãŒãæ€çŽ¢ã®æåã®ããŒãžã衚瀺ããŸãã |
次ã®ããŒãžããã£ãã·ã¥ãã | æ°ã«ããªã | ãã£ãã·ã¥ã«ç¶ããŠèŠããŠãã ããã |
次ã®ããŒãžããã£ãã·ã¥ãªã | ããŒãã« | ããäžåºŠæ€çŽ¢ããŠåãå ŽæããèŠããŠãã ããã |
次ã®ããŒãžããã£ãã·ã¥ãªã | 倱æ | ããã§ãªã¹ããçµäºããŸãã |
äœä»æ§ãµãŒããŒ/ç¡æå¶çŽãå æããããã®èœå
æ€çŽ¢ã¯2ç§ãŸã§åŸ ã¡ãŸããæž¡ããšããã«ããŒã¯ãŒãæ€çŽ¢ã«é²ã¿ãå詊è¡ããŸããã
æ¥ç¶ã§ããªããšãã°ããäŒã¿ãŸãã 30ç§ããå§ããŠæå€§6æéãŸã§åæ¥ç¶ããŸãããã¯ã©ã¹ã¿ãç ãã«èœã¡ããšãã«ããã¹ãŠã®èŠæ±ã2ç§ãã€åŸ ã€ã®ãé²ããŸããåã€ã³ããã¯ã¹ãå®è¡ãããšããã«åæ¥ç¶ããŸãã
æåã®åŒã³åºãã ããããäžåºŠè©ŠããŠãã ããããµãŒããŒããªã³ã«ãªã£ããšããæåã®gRPCåŒã³åºãã¯æºåæéã®ãã2ç§ãè¶ ãããããªããŸããããã§ãããçŽããŠãåãçµæã®æžãèŸŒã¿æäœãããäžåºŠè©ŠããŠãã ããã
äžæ¥ã®äœ¿çšéãåããŸãããæ€çŽ¢APIã¯èª°ã§ãåŒã³åºãããšãã§ããŸãã誰ããæ€çŽ¢èªãå€ãç¶ãããšãäžæ¥ã®åã蟌ã¿å¶éãäžãããã€ã³ããã¯ã¹ãŸã§æ¢ãŸããŸããã ãããæ€çŽ¢ã¯1æ¥1,900ä»¶ãŸã§ãã䜿ããŸãããã100ä»¶ã¯ã€ã³ããã¯ã¹ã·ã§ã¢ãšããŠæ®ããŸãããã£ãã·ã¥ã§ããã«èŠã€ããæ€çŽ¢ã¯æ°ããŸããã䜿çšéãæ°ããRedisãæ¢ãŸããšæ€çŽ¢ã¯æ¢ãŸãããã®ãŸãŸé²ã¿ãŸãã

æ¯æ¥ã¯ã©ã¹ã¿ãç®èŠããããŸãã ã7æ¥ä»¥å ã«æžããã忢ããé¿ããããšæ¯æ¥å€æãã«è»œãåŒã³åºããéããŸãããã ãããã®åŒã³åºããå®éã«åæ¢ãé²ããã©ããã¯ãŸã 確èªããŠããŸããã
ã¹ã€ããã§ãªã³/ãªãã§ããŸãã
çµãã
ã¹ã¿ãã£ãããŠãã¯ã¿ãŒåã蟌ã¿ãäœãªã®ããã©ã®ããã«æ¢ããŠããã®ããããããèŠã€ããããããã«ããã«ã¯ãäœã調ç¯ããŠããã®ã確èªããå¿ èŠãããã®ããªã©ãåŠãã ããšããããŸããã ããaiã䜿ã£ãŠçè§£ããããšã«åºã¥ããŠãã¯ãã«æ€çŽ¢ã§ããããã«è¿œå ããã®ã§ããã®ããã«å®è£ ãé£ãããããŸããã§ãããããããå®éã«äœ¿ã£ãŠã¿ããšãã®æºè¶³åºŠã¯ãŸã ããããããŸããããã¡ããäœããã£ãšè€æ°ãæ€çŽ¢ãããæ¬åœã«é¢é£ããŠããããã«èŠããã®ãåºãŠããã®ã§ããã
