NVIDIA NeMo Framework Tus Neeg Siv Qhia

NVIDIA NeMo Framework

Specifications

• Khoom npe: NVIDIA NeMo Framework
• Kev cuam tshuam Platforms: Windows, Linux, macOS
• Qhov cuam tshuam Versions: Tag nrho cov versions ua ntej 24
• Kev ruaj ntseg Vulnerability: CVE-2025-23360
• Risk Assessment Base Score: 7.1 (CVSS v3.1)

Cov lus qhia siv khoom

Kev Ruaj Ntseg Hloov Kho Kev Txhim Kho:
Txhawm rau tiv thaiv koj lub cev, ua raws li cov kauj ruam no:

1. Download tau qhov tseeb tso tawm los ntawm NeMo-Framework-Launcher Tso Tawm nplooj ntawv ntawm GitHub.
2. Mus rau NVIDIA Product Security yog xav paub ntxiv.

Kev ruaj ntseg hloov tshiab cov ntsiab lus:
Qhov hloov tshiab kev ruaj ntseg hais txog qhov tsis zoo hauv NVIDIA NeMo Framework uas tuaj yeem ua rau kev ua tiav code thiab cov ntaub ntawv tampkev pom.

Txawj tej yam ntxiv Software:
Yog tias koj tab tom siv cov ceg tso tawm ua ntej, nws raug nquahu kom hloov mus rau cov ceg tshiab tso tawm tshiab los daws qhov teeb meem kev nyab xeeb.

Tshajview

NVIDIA NeMo Ncej yog qhov ua kom muaj peev xwm thiab huab-native generative AI lub moj khaum tsim rau cov kws tshawb fawb thiab cov neeg tsim khoom ua haujlwm ntawm Cov qauv lus loj, Multimodal, thiab Hais lus AI (eg Tsis siv neeg hais lus paub thiab Lus-rau-hais lus). Nws tso cai rau cov neeg siv los tsim, kho kom zoo, thiab xa cov qauv AI tshiab los ntawm kev siv cov cai uas twb muaj lawm thiab cov qauv kuaj xyuas ua ntej.

Teeb cov lus qhia: Nruab NeMo Framework

Cov qauv lus loj thiab Multimodal qauv
NeMo Framework muab kev txhawb nqa kawg rau kev tsim Cov Qauv Loj (LLMs) thiab Multimodal Models (MMs). Nws muab qhov yooj yim los siv rau ntawm thaj chaw, hauv cov ntaub ntawv-chaw, lossis nrog koj tus kws kho mob nyiam huab. Nws kuj tseem txhawb nqa kev ua tiav ntawm SLURM lossis Kubernetes qhov chaw qhib.

Kev kho cov ntaub ntawv
NeMo Curator [1] yog lub tsev qiv ntawv Python uas suav nrog cov suite ntawm modules rau cov ntaub ntawv mining thiab tsim cov ntaub ntawv hluavtaws. Lawv yog cov scalable thiab optimized rau GPUs, ua rau lawv zoo tagnrho rau curating natural language data to train or fine-tune LLMs. Nrog NeMo Curator, koj tuaj yeem tshem tawm cov ntawv zoo los ntawm cov ntaub ntawv nyoos web cov ntaub ntawv.

Kev cob qhia thiab Customization

NeMo Framework muab cov cuab yeej rau kev cob qhia zoo thiab kev hloov kho ntawm LLMs thiab Multimodal qauv. Nws suav nrog cov kev teeb tsa ua ntej rau kev teeb tsa hauv pawg, rub tawm cov ntaub ntawv, thiab cov qauv hyperparameters, uas tuaj yeem hloov kho rau kev cob qhia ntawm cov ntaub ntawv tshiab thiab cov qauv. Ntxiv rau kev cob qhia ua ntej, NeMo txhawb nqa ob qho tib si Saib Xyuas Kev Pom Zoo-Tuning (SFT) thiab Parameter Efficient Fine-Tuning (PEFT) cov txheej txheem xws li LoRA, Ptuning, thiab ntau dua.

Ob txoj kev xaiv muaj los tsim kev cob qhia hauv NeMo - siv NeMo 2.0 API interface lossis nrog NeMo Run.

• Nrog NeMo Run (pom zoo): NeMo Run muab kev sib txuas rau kev txhim kho kev teeb tsa, kev ua tiav thiab kev tswj hwm kev sim thoob plaws ntau qhov chaw suav. Qhov no suav nrog kev pib ua haujlwm ntawm koj qhov chaw ua haujlwm hauv zos lossis hauv pawg loj - ob qho tib si SLURM enabled lossis Kubernetes hauv huab huab.
  • Kev cob qhia ua ntej & PEFT Quickstart nrog NeMo Khiav
• Siv NeMo 2.0 API: Txoj kev no ua haujlwm zoo nrog kev teeb tsa yooj yim uas muaj cov qauv me me, lossis yog tias koj xav sau koj tus kheej cov ntaub ntawv kev cai, kev cob qhia loops, lossis hloov cov qauv txheej. Nws ua rau koj yooj yim dua thiab tswj cov kev teeb tsa, thiab ua rau nws yooj yim rau txuas ntxiv thiab kho cov teeb tsa hauv programmatically.
  • TraIning Quickstart nrog NeMo 2.0 API
  • Migrating los ntawm NeMo 1.0 rau NeMo 2.0 API

Kev sib tw

• NeMo-Aligner [1] yog ib lub cuab yeej uas muaj peev xwm ua kom tau raws li tus qauv zoo. Cov cuab yeej muaj kev txhawb nqa rau lub xeev-of-the-art qauv algorithms xws li SteerLM, DPO, Kev Tshawb Fawb Txog Kev Tshawb Fawb los ntawm Tib Neeg Cov Lus Teb (RLHF), thiab ntau ntxiv. Cov algorithms no tso cai rau cov neeg siv los kho cov qauv lus kom muaj kev nyab xeeb dua, tsis muaj mob, thiab pab tau.
• Tag nrho NeMo-Aligner checkpoints yog hla-tshaj nrog NeMo ecosystem, tso cai rau kev hloov kho ntxiv thiab kev xa tawm.

Kev ua haujlwm ib ntus ntawm tag nrho peb theem ntawm RLHF ntawm tus qauv GPT-2B me me:

• Kev cob qhia SFT
• Khoom plig qauv kev cob qhia
• Kev cob qhia PPO

Tsis tas li ntawd, peb ua qauv qhia kev txhawb nqa rau ntau yam kev sib tw tshiab:

• DPO: ib lub teeb yuag algorithm piv rau RLHF nrog kev ua haujlwm yooj yim dua.
• Ua si tus kheej Fine-Tuning (SPIN)
• SteerLM: ib tug txheej txheem raws li conditioned-SFT, nrog steerable tso zis.

Mus saib cov ntaub ntawv kom paub ntau ntxiv: Cov ntaub ntawv sib dhos

Multimodal Qauv

• NeMo Framework muab cov software zoo tshaj plaws los cob qhia thiab siv cov qauv ua qauv ntawm lub xeev-ntawm-the-art multimodal nyob rau ntau pawg: Multimodal Language Models, Vision-Language Foundations, Text-to-Image model, thiab dhau 2D Generation siv Neural Radiance Fields (NeRF).
• Txhua pawg yog tsim los ua kom tau raws li cov kev xav tau tshwj xeeb thiab kev nce qib hauv thaj chaw, siv cov qauv txiav-ntug los tswj ntau hom ntaub ntawv, suav nrog cov ntawv nyeem, duab, thiab 3D qauv.

Kev xa tawm thiab kev xav
NeMo Framework muab ntau txoj hauv kev rau LLM inference, catering rau cov xwm txheej sib txawv thiab kev xav tau kev ua haujlwm.

Deploy with NVIDIA NIM

• NeMo Framework seamlessly integrates nrog enterprise-level qauv xa cov cuab yeej los ntawm NVIDIA NIM. Qhov kev sib koom ua ke no yog siv los ntawm NVIDIA TensorRT-LLM, kom ntseeg tau tias qhov ua tau zoo thiab muaj peev xwm ntsuas tau.
• Yog xav paub ntxiv txog NIM, mus saib NVIDIA webqhov chaw.

Tso nrog TensorRT-LLM lossis vLLM

• NeMo Framework muaj cov ntawv sau thiab APIs los xa cov qauv mus rau ob qhov kev xav tau zoo ntawm cov tsev qiv ntawv, TensorRT-LLM thiab vLLM, thiab xa cov qauv xa tawm nrog NVIDIA Triton Inference Server.
• Rau cov xwm txheej uas xav tau kev ua tau zoo, NeMo qauv tuaj yeem siv TensorRT-LLM, lub tsev qiv ntawv tshwj xeeb rau kev nrawm thiab ua kom zoo dua LLM kev xav ntawm NVIDIA GPUs. Cov txheej txheem no suav nrog kev hloov cov qauv NeMo rau hauv hom sib xws nrog TensorRT-LLM siv nemo.export module.
  • LLM Deployment Overview
  • Deploy NeMo Cov Qauv Lus Loj nrog NIM
  • Deploy NeMo Cov Qauv Lus Loj nrog TensorRT-LLM
  • Deploy NeMo Cov Qauv Lus Loj nrog vLLM

Txhawb Cov Qauv

Cov qauv lus loj

Cov qauv lus loj

Cov qauv lus loj	Pretraining & SFT	PEFT	Kev sib tw	FP8 Kev cob qhia Convergence	TRT/TRTLLM	Hloov rau & Los ntawm puag lub ntsej muag	Kev ntsuam xyuas
Llama3 8B/70B, Llama3.1 405B	Yog lawm	Yog lawm	x	Yog (ib nrab txheeb xyuas)	Yog lawm	Ob leeg	Yog lawm
Mixtral 8x7B / 8x22B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	Yog lawm	Ob leeg	Yog lawm
Nemotron 3 8B	Yog lawm	x	x	Yog (tsis tau lees paub)	x	Ob leeg	Yog lawm
Nemotron 4 340B	Yog lawm	x	x	Yog (tsis tau lees paub)	x	Ob leeg	Yog lawm
Baichuan 2 7B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	x	Ob leeg	Yog lawm
ChatGLM3 6B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	x	Ob leeg	Yog lawm
Gemma 2B/7B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	Yog lawm	Ob leeg	Yog lawm
Gemma2 2B/9B/27B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	x	Ob leeg	Yog lawm
Mamba2 130M/370M/780M/1.3B/2.7B/8B/ Hybrid-8B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	x	x	Yog lawm
Phi 3 mini 4k	x	Yog lawm	x	Yog (tsis tau lees paub)	x	x	x
Qwen2 0.5B/1.5B/7B/72B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	Yog lawm	Ob leeg	Yog lawm
StarCoder 15B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	Yog lawm	Ob leeg	Yog lawm
StarCoder2 3B/7B/15B	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	Yog lawm	Ob leeg	Yog lawm
BERT 110M / 340M	Yog lawm	Yog lawm	x	Yog (tsis tau lees paub)	x	Ob leeg	x
T5 220M/3B/11B	Yog lawm	Yog lawm	x	x	x	x	x

 

Vision Language Models

 

Cov qauv tsim

 

Ntiaj teb Foundation Models

Nco tseg
NeMo kuj txhawb nqa pretraining rau ob qho tib si diffusion thiab autoregressive architectures text2world foundation qauv.

Hais lus AI

Tsim cov qauv kev sib tham AI yog cov txheej txheem nyuaj uas suav nrog kev txhais, tsim, thiab kev cob qhia cov qauv hauv qee thaj chaw. Cov txheej txheem no feem ntau xav tau ntau qhov kev rov ua dua kom ncav cuag qib siab ntawm qhov raug. Nws feem ntau suav nrog ntau qhov kev hloov pauv kom ua tiav qhov raug siab, kho kom zoo ntawm ntau yam haujlwm thiab cov ntaub ntawv tshwj xeeb, ua kom muaj kev cob qhia kev ua haujlwm, thiab npaj cov qauv rau kev xa tawm.

NeMo Framework muab kev txhawb nqa rau kev cob qhia thiab kev hloov kho ntawm Speech AI qauv. Qhov no suav nrog cov haujlwm xws li Tsis Siv Neeg Speech Recognition (ASR) thiab Text-To-Speech (TTS) synthesis. Nws muaj kev hloov pauv mus rau kev lag luam qib kev xa tawm nrog NVIDIA Riva. Txhawm rau pab cov neeg tsim khoom thiab cov kws tshawb fawb, NeMo Framework suav nrog cov chaw kuaj xyuas ua ntej kev cob qhia hauv lub xeev, cov cuab yeej rau kev tsim cov ntaub ntawv hais lus rov ua dua, thiab cov yam ntxwv rau kev sib tham sib tshawb nrhiav thiab tshuaj xyuas cov ntaub ntawv hais lus. Cov Cheebtsam ntawm NeMo Ncej rau Kev Hais Lus AI yog raws li hauv qab no:

Kev cob qhia thiab Customization
NeMo Framework muaj txhua yam xav tau los cob qhia thiab kho cov qauv hais lus (ASR, Kev faib lus, Hais Lus Paub, Hais lus Diarization, thiab TTS) nyob rau hauv ib tug reproductible.

SOTA Pre-kawm qauv

• NeMo Framework muab cov zaub mov tseem ceeb hauv lub xeev thiab cov chaw kuaj xyuas ua ntej ntawm ob peb ASR thiab TTS cov qauv, nrog rau cov lus qhia yuav ua li cas thauj lawv.
• Cov cuab yeej hais lus
• NeMo Framework muab cov cuab yeej siv tau los tsim ASR thiab TTS qauv, suav nrog:
  • NeMo Forced Aligner (NFA) rau generating token-, lo lus- thiab segment-theem timestamps ntawm kev hais lus hauv suab siv NeMo's CTC-based Automatic Speech Recognition qauv.
  • Speech Data Processor (SDP), cov cuab yeej siv los ua kom yooj yim hais lus cov ntaub ntawv. Nws tso cai rau koj los sawv cev cov ntaub ntawv ua haujlwm hauv kev teeb tsa file, minimizing boilerplate code thiab tso cai reproducibility thiab shareability.
  • Speech Data Explorer (SDE), Dash-based web daim ntawv thov kev sib tham sib tshawb nrhiav thiab tshuaj xyuas cov ntaub ntawv hais lus.
  • Dataset creation tool uas muab kev ua haujlwm los ua kom lub suab ntev files nrog rau cov ntawv sau sib raug thiab muab faib ua cov ntu luv uas tsim nyog rau Kev Kawm Hais Lus Tsis Siv Neeg (ASR) qauv.
  • Cov cuab yeej sib piv rau ASR Cov Qauv los sib piv cov kev kwv yees ntawm cov qauv ASR sib txawv ntawm cov lus raug thiab hais lus.
  • ASR Evaluator rau kev soj ntsuam kev ua tau zoo ntawm ASR qauv thiab lwm yam nta xws li Kev Tshawb Fawb Lub Suab.
  • Text Normalization Tool rau kev hloov cov ntawv los ntawm daim ntawv sau rau hauv daim ntawv hais lus thiab rov ua dua (piv txwv li "31st" vs "peb caug thawj").
• Txoj kev mus rau Deployment
• NeMo cov qauv uas tau txais kev cob qhia lossis kho kom haum siv NeMo Framework tuaj yeem ua kom zoo thiab siv nrog NVIDIA Riva. Riva muab cov thawv ntim thiab Helm kab kos tshwj xeeb tsim los ua kom cov kauj ruam rau kev xa mus rau lub laub khawm.

Lwm Cov Kev Pabcuam

GitHub Repos

• NeMo: Lub chaw khaws ntaub ntawv tseem ceeb rau NeMo Framework
• NeMo-Khiav: Ib qho cuab yeej los teeb tsa, tso tawm thiab tswj koj cov kev sim kev kawm tshuab.
• NeMo-Aligner: Scalable toolkit rau cov qauv zoo sib xws
• NeMo-Curator: Scalable data pre-processing and curation toolkit rau LLMs

Tau txais kev pab
Koom nrog NeMo zej zog, nug cov lus nug, tau txais kev txhawb nqa, lossis tshaj tawm cov kab mob.

• NeMo Kev Sib Tham
• NeMo teeb meem

Programming Languages ​​thiab Frameworks

• Python: Lub ntsiab interface siv NeMo Framework
• Pytorch: NeMo Framework yog ua rau saum PyTorch

Daim ntawv tso cai

• NeMo Github repo muaj ntawv tso cai raws li Apache 2.0 daim ntawv tso cai
• NeMo Framework tau tso cai raws li NVIDIA AI PRODUCT Agreement. Los ntawm rub thiab siv lub thawv, koj lees txais cov nqe lus thiab cov cai ntawm daim ntawv tso cai no.
• Lub thawv NeMo Framework muaj Llama cov ntaub ntawv tswj hwm los ntawm Meta Llama3 Daim Ntawv Pom Zoo rau Zej Zog.

Cov ntawv hauv qab
Tam sim no, NeMo Curator thiab NeMo Aligner txhawb nqa rau Multimodal qauv yog kev ua haujlwm thiab yuav muaj sai sai.

FAQ

Q: Kuv tuaj yeem tshawb xyuas seb kuv lub cev puas cuam tshuam los ntawm qhov tsis zoo?
A: Koj tuaj yeem tshawb xyuas seb koj lub kaw lus puas cuam tshuam los ntawm kev txheeb xyuas qhov version ntawm NVIDIA NeMo Framework ntsia. Yog tias nws qis dua version 24, koj lub kaw lus yuav muaj qhov tsis zoo.

Q: Leej twg qhia qhov teeb meem kev ruaj ntseg CVE-2025-23360?
A: Qhov teeb meem kev ruaj ntseg tau tshaj tawm los ntawm Or Peles - JFrog Security. NVIDIA lees paub lawv qhov kev koom tes.

Q: Kuv tuaj yeem tau txais cov ntawv ceeb toom kev ruaj ntseg yav tom ntej li cas?
A: Mus saib nplooj ntawv NVIDIA Khoom Ruaj Ntseg txhawm rau sau npe rau cov ntawv ceeb toom kev nyab xeeb thiab nyob twj ywm paub txog cov khoom lag luam ruaj ntseg hloov tshiab.