Hugging Face Transformers

Hugging Face Transformers ist derzeit die beliebteste Open-Source-NLP-/KI-Bibliothek und bietet Tausende vortrainierte Modelle, die nahezu alle KI-Aufgaben abdecken, einschließlich Text, Bild, Audio und multimodaler Aufgaben.

Der Kernwert besteht darin, komplexe Modelllade-, Inferenz- und Trainingsprozesse in wenigen Codezeilen zu kapseln.

Hugging Face Ecosystem Overview Hub 400K+ Models 100K+ Datasets Transformers Pretrained Model Inference and Fine-tuning Framework Datasets Massive Datasets Efficient Loading and Processing PEFT Parameter-Efficient Fine-Tuning LoRA / QLoRA Accelerate Multi-GPU/TPU Training Distributed Acceleration Tokenizers High-Performance Tokenizer (Rust implementation) Evaluate Model Evaluation Metrics (BLEU/F1, etc.)

Unterstützte Aufgabentypen

Task Categories Supported by Transformers NLP Natural Language Processing Text Classification (Sentiment Analysis) Named Entity Recognition (NER) Question Answering (QA) Text Summarization Machine Translation Text Generation (Dialogue) Fill-Mask / Language Modeling CV Computer Vision Image Classification Object Detection Image Segmentation Depth Estimation Image Generation Video Classification Keypoint Detection Audio & Multimodal Speech Recognition (ASR) Audio Classification Text-to-Speech (TTS) Image-Text Matching (VQA) Image Captioning Document Question Answering (Doc QA) Zero-Shot Classification

Kernprinzipien der Transformer-Architektur

Bevor Sie die Bibliothek verwenden, können Sie durch das Verständnis der zugrunde liegenden Architektur nachvollziehen, warum Sie Parameter auf diese Weise abstimmen.

Gesamtarchitektur: Encoder-Decoder

Drei große Modellfamilien

Three Major Model Families: Architecture x Use Cases x Representative Models Encoder-Only Models Bidirectional attention, understands global context Suitable for: Classification, NER, QA (understanding tasks) Representatives: BERT, RoBERTa, ALBERT Chinese: BERT-wwm, MacBERT Feature: Can see all inputs simultaneously Input -> [CLS] represents global, [MASK] for pretraining Decoder-Only Models Causal attention (only looks left), autoregressive generation Suitable for: Text generation, dialogue, code generation Representatives: GPT series, LLaMA, Qwen Chinese: ChatGLM, Baichuan Feature: Predicts the next word one by one Input -> predict next token -> concatenate and predict again -> loop Encoder-Decoder model Sequence-to-sequence (Seq2Seq) task Suitable for: translation, summarization, question-answer generation Representatives: T5, BART, mT5 Chinese: mT5, PEGASUS-Chinese Features: encode input, decode to generate output Source sequence -> Encoder -> Decoder -> Target sequence

Installation und Umgebungskonfiguration

Installation

# 基础安装
pip install transformers

# 完整安装（包含训练依赖）
pip install transformers[torch]       # PyTorch 后端（推荐）
pip install transformers[tf-cpu]      # TensorFlow 后端
pip install transformers[flax]        # JAX/Flax 后端

# 常用配套库
pip install datasets          # HuggingFace 数据集库
pip install evaluate          # 模型评估指标
pip install accelerate        # 多GPU/混合精度训练
pip install peft              # 参数高效微调（LoRA等）
pip install tokenizers        # 高性能分词器
pip install sentencepiece     # 部分模型（T5/LLaMA）需要

# 验证安装
python -c "import transformers; print(transformers.__version__)"

Konfiguration von Umgebungsvariablen

# 设置模型缓存目录（模型下载后缓存到此路径，默认 ~/.cache/huggingface）
export HF_HOME=/data/huggingface_cache

# 国内用户：使用镜像站加速下载（推荐 hf-mirror.com）
export HF_ENDPOINT=https://hf-mirror.com

# 离线模式（网络不可用时，只使用已缓存的模型）
export TRANSFORMERS_OFFLINE=1

# 禁用进度条（CI/CD 环境）
export DISABLE_TQDM=1

Beispiel

# Kann auch im Code festgelegt werden
import os
os.environ["HF_ENDPOINT"] = "https://hf-mirror.com"

# Aktuelles Cache-Verzeichnis anzeigen
from transformers.utils import TRANSFORMERS_CACHE
print(TRANSFORMERS_CACHE)

Pipeline: KI mit fünf Codezeilen ausführen

Pipeline ist die höchste Abstraktionsebene in Transformers. Sie kapselt Modellladen, Vorverarbeitung, Inferenz und Nachverarbeitung vollständig und ermöglicht es Ihnen, die Inferenz in nur drei bis fünf Codezeilen abzuschließen.

Pipeline internal workflow Raw input Text / image Audio / other data Preprocessing Tokenizer tokenization Convert to token IDs Padding/Truncation Model inference Forward pass Output logits or hidden state vectors Post-processing Softmax/Argmax Decode tokens -> text Format output Result Label/score Generated text Pipeline automatically completes all steps; users only need to pass in raw input and get formatted results

Sammlung schneller Pipeline-Beispiele

Beispiel

from transformers import pipeline

# 1. Sentimentanalyse (Textklassifikation)
classifier = pipeline("sentiment-analysis")
result = classifier("I love using Hugging Face Transformers!")
# -> [{'label': 'POSITIVE', 'score': 0.9998}]

# 2. Textgenerierung
generator = pipeline("text-generation", model="gpt2")
result = generator("Once upon a time in a land far away,",
    max_new_tokens=50, num_return_sequences=1, temperature=0.8)

# 3. Lückentext (maskiertes Sprachmodell)
unmasker = pipeline("fill-mask", model="bert-base-uncased")
result = unmasker("The capital of France is [MASK].")
# -> [{'token_str': 'paris', 'score': 0.9823}, ...]

# 4. Erkennung benannter Entitäten (NER)
ner = pipeline("ner", aggregation_strategy="simple")
result = ner("My name is John and I work at Google in New York.")
# -> [{'entity_group': 'PER', 'word': 'John', 'score': 0.998}, ...]

# 5. Extraktive Fragebeantwortung
qa = pipeline("question-answering")
result = qa(question="Who invented Python?",
    context="Python was created by Guido van Rossum in 1991.")
# -> {'answer': 'Guido van Rossum', 'score': 0.9887}

# 6. Textzusammenfassung
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
result = summarizer(article, max_length=60, min_length=20)

# 7. Maschinelle Übersetzung
translator = pipeline("translation", model="Helsinki-NLP/opus-mt-en-zh")
result = translator("Hello, how are you today?")
# -> [{'translation_text': 'Hello, how are you today?'}]

# 8. Zero-Shot-Klassifikation (kein spezielles Training erforderlich)
zero_shot = pipeline("zero-shot-classification")
result = zero_shot("I love playing football",
    candidate_labels=["sports", "politics", "technology"])
# -> {'labels': ['sports', ...], 'scores': [0.972, ...]}

Erweiterte Pipeline-Konfiguration

Beispiel

import torch
from transformers import pipeline

# GPU angeben
pipe = pipeline("text-generation", model="gpt2", device=0)

# Präzision angeben (Speicher sparen)
pipe = pipeline("text-generation", model="meta-llama/Llama-2-7b-hf",
    torch_dtype=torch.float16, device_map="auto")

# Stapelverarbeitung (Durchsatz erhöhen)
pipe = pipeline("sentiment-analysis", batch_size=32)
results = pipe(large_text_list)    # Automatische Stapel-Inferenz

# Segmentierung großer Texte
asr = pipeline("automatic-speech-recognition",
    model="openai/whisper-large-v2",
    chunk_length_s=30, stride_length_s=5)
result = asr("long_audio.wav", return_timestamps=True)

Detaillierte Analyse des Tokenizers

Tokenizer ist der erste Schritt in NLP: Umwandlung von Rohtext in eine Sequenz von Zahlen, die das Modell verstehen kann.

Vollständiger Tokenisierungsprozess

Complete Tokenization Process: Text -> Model Input Step 1 Raw Text: "Hello, I'm learning Transformers! It's great." Step 2 Tokenize: ["Hello", ",", "I", "'m", "learning", "Transform", "##ers", "!", ...] WordPiece/BPE subword tokenization: rare words are split (Transformers -> Transform + ##ers) Step 3 Add Special Tokens: ["[CLS]", "Hello", ",", "I", "'m", "learning", "Transform", "##ers", ... "[SEP]"] [CLS] classification token, [SEP] separator, [PAD] padding; different models have different special tokens Step 4 Convert to Token IDs: [101, 7592, 1010, 1045, 1005, 1049, 4083, 19081, 2121, ... 102] Each token maps to an integer index in the vocabulary and is fed into the model's Embedding layer.

Grundlegende Verwendung des Tokenizers

Beispiel

from transformers import AutoTokenizer

# Tokenizer laden
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# In einem Schritt kodieren
encoding = tokenizer(
    "Hello, I'm learning Transformers!",
    return_tensors="pt",        # PyTorch-Tensor zurückgeben
    padding=True,               # Auf die längste Sequenz auffüllen
    truncation=True,            # Bei Überschreitung der Länge abschneiden
    max_length=128,             # Maximale Länge
)

print(encoding.keys())
# -> dict_keys(['input_ids', 'token_type_ids', 'attention_mask'])

print(encoding["input_ids"][0][:8])
# -> tensor([101, 7592, 1010, 1045, 1005, 1049, 4083, 19081])

print(encoding["attention_mask"][0][:8])
# -> tensor([1, 1, 1, 1, 1, 1, 1, 1]) # 1=reales Token, 0=Padding

# Dekodieren (ID -> Text)
decoded = tokenizer.decode(encoding["input_ids"][0], skip_special_tokens=True)
print(decoded)   # -> "Hallo, ich lerne Transformers!"

# Batch-Kodierung (automatische Padding-Ausrichtung)
texts = ["Kurz.", "Dies ist ein viel längerer Satz zum Testen."]
batch = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
print(batch["input_ids"].shape)   # -> torch.Size([2, 10])

# Vokabular-Informationen
print(f"Vokabulargröße: {tokenizer.vocab_size}")      # -> 30522
print(f"[CLS]-ID: {tokenizer.cls_token_id}")     # -> 101
print(f"[SEP]-ID: {tokenizer.sep_token_id}")     # -> 102
print(f"Maximale Länge: {tokenizer.model_max_length}")  # -> 512

Vergleich gängiger Tokenizer-Typen

Comparison of Three Mainstream Tokenization Algorithms Algorithm Principle Example Tokenization Representative Model BPE Byte Pair Encoding Merge high-frequency character pairs Learn optimal subword vocabulary "transformers" -> ["transform", "ers"] GPT / RoBERTa WordPiece Maximize language model probability ## Prefix-marked subwords "transformers" -> ["transform", "##ers"] BERT / DistilBERT SentencePiece Unigram / BPE variants Language-independent, direct processing Raw bytes, leading markers indicate word starts "transformers" -> ["_transform", "ers"] T5 / LLaMA / Qwen

Modellladen und Inferenz

AutoClass: Automatische Auswahl der richtigen Modellklasse

How AutoClass works: automatically matches the correct model architecture Model Name "bert-base-uncased" or local path AutoClass Load config.json Match architecture type AutoModel / AutoTokenizer / ... Automatically return the correct class BertForSequenceClassification GPT2LMHeadModel T5ForConditionalGeneration ... Common AutoClass quick reference: AutoTokenizer Auto tokenizer AutoModel Base model (outputs hidden states) AutoModelForSeqClass Text classification task AutoModelForCausalLM Text generation task

Beispiel

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
    model_name, num_labels=2, torch_dtype=torch.float16, device_map="auto"
)

# Vollständiger manueller Inferenzprozess
text = "Transformers ist eine großartige Bibliothek!"

# 1. Kodierung
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
inputs = {k: v.to(model.device) for k, v in inputs.items()}

# 2. Vorwärtspropagierung
with torch.no_grad():
    outputs = model(**inputs)

# 3. Ausgabe analysieren
logits = outputs.logits                       # Form: [1, 2]
probs  = torch.softmax(logits, dim=-1)
pred   = torch.argmax(probs, dim=-1).item()

id2label = model.config.id2label              # {0: 'LABEL_0', 1: 'LABEL_1'}
print(f"Vorhergesagte Klasse: {id2label[pred]}, Konfidenz: {probs[0][pred]:.4f}")

Extraktion von Satzvektoren

Beispiel

from transformers import AutoModel, AutoTokenizer
import torch

model = AutoModel.from_pretrained("bert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

def get_sentence_embedding(text: str) -> torch.Tensor:
    inputs = tokenizer(text, return_tensors="pt", max_length=512, truncation=True)
    with torch.no_grad():
        outputs = model(**inputs)
    # Mean-Pooling auf alle Tokens anwenden
    token_embeddings = outputs.last_hidden_state          # [1, seq_len, 768]
    attention_mask = inputs["attention_mask"].unsqueeze(-1)
    mean_embedding = (token_embeddings * attention_mask).sum(1) / attention_mask.sum(1)
    return mean_embedding  # [1, 768]

vec = get_sentence_embedding("Hallo Welt")
print(vec.shape)  # -> torch.Size([1, 768])

Praktischer Leitfaden für zehn häufige Aufgaben

Textklassifikation (Sentimentanalyse)

Beispiel

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "cardiffnlp/twitter-roberta-base-sentiment-latest"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

def predict_sentiment(texts):
    inputs = tokenizer(texts, return_tensors="pt", padding=True,
                       truncation=True, max_length=512)
    with torch.no_grad():
        logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)
    results = []
    for i, t in enumerate(texts):
        pid = probs[i].argmax().item()
        results.append({"text": t, "label": model.config.id2label[pid],
                         "score": round(probs[i][pid].item(), 4)})
    return results

print(predict_sentiment(["Ich liebe das!", "Das ist schrecklich."]))
# -> [{'text': 'Ich liebe das!', 'label': 'positiv', 'score': 0.9756}, ...]

Textgenerierung (Dialog / Fortsetzung)

Beispiel

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "Qwen/Qwen2-1.5B-Instruct"     # Alibaba Tongyi Qianwen (unterstützt Chinesisch)
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.float16, device_map="auto"
)

messages = [
    {"role": "system", "content": "Du bist ein hilfreicher KI-Assistent."},
    {"role": "user", "content": "Bitte erkläre in drei Sätzen, was ein Transformer ist."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False,
                                      add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        **inputs, max_new_tokens=300, temperature=0.7, top_p=0.9,
        do_sample=True, repetition_penalty=1.1,
        pad_token_id=tokenizer.eos_token_id,
    )

new_tokens = output_ids[0][inputs["input_ids"].shape[1]:]
response = tokenizer.decode(new_tokens, skip_special_tokens=True)
print(response)

Erkennung benannter Entitäten (NER)

Beispiel

from transformers import pipeline

ner = pipeline("ner", model="dslim/bert-base-NER", aggregation_strategy="simple")
result = ner("Elon Musk gründete 2002 SpaceX und 2003 Tesla Motors.")
for entity in result:
    print(f"{entity['word']:<20} -> {entity['entity_group']} ({entity['score']:.3f})")

# Chinesische NER
ner_cn = pipeline("ner", model="hfl/chinese-bert-wwm-ext-ner-msra",
                   aggregation_strategy="simple")
result = ner_cn("Xiao Ming studierte an der Peking-Universität und arbeitete später bei Alibaba.")

Maschinelle Übersetzung

Beispiel

from transformers import pipeline

# Englisch nach Chinesisch
translator = pipeline("translation", model="Helsinki-NLP/opus-mt-en-zh")
result = translator("Künstliche Intelligenz verändert die Welt.")
print(result[0]["translation_text"])   # -> Künstliche Intelligenz verändert die Welt.

# Chinesisch nach Englisch
translator_zh = pipeline("translation", model="Helsinki-NLP/opus-mt-zh-en")
result = translator_zh("Künstliche Intelligenz verändert die Welt.")
print(result[0]["translation_text"])

Textzusammenfassung

Beispiel

from transformers import pipeline

summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
result = summarizer(long_text, max_length=80, min_length=30,
                     do_sample=False, no_repeat_ngram_size=3)
print(result[0]["summary_text"])

Fine-Tuning

Fine-Tuning passt ein vortrainiertes Modell an Ihre spezifische Aufgabe und Ihre Daten an und ist das wichtigste Anwendungsszenario von Transformers.

Überblick über den Fine-Tuning-Prozess

Complete fine-tuning process with Transformers Prepare data Dataset loading HF datasets or custom CSV Tokenize encoding Padding alignment DataLoader Key: data quality > Data quantity Load model AutoModelForTask Specify num_labels Freeze bottom-layer parameters (optional) Only fine-tune the top layers Tip: small dataset Freeze the bottom layers first Training configuration TrainingArguments Learning rate: 2e-5 Batch：32 Epochs：3~5 Warmup steps Weight decay Tip: Learning rate Most important Training & Evaluation Trainer.train() Monitor loss curve Validation set evaluation Early stopping Save best ckpt Note: Overfitting Compare baseline Save save_model() save_pretrained Push to Hub Production deployment Quantized inference ONNX export Using HuggingFace Trainer can greatly simplify the training pipeline, automatically handling gradient accumulation, mixed precision, distributed training, etc.

Vollständiges Fine-Tuning-Beispiel: Textklassifikation

Beispiel

from datasets import load_dataset
from transformers import (
    AutoTokenizer, AutoModelForSequenceClassification,
    TrainingArguments, Trainer, DataCollatorWithPadding,
    EarlyStoppingCallback,
)
import evaluate, numpy as np

# 1. Datensatz laden
dataset = load_dataset("imdb")   # Öffentlicher HF-Hub-Datensatz

# 2. Tokenizer + Vorverarbeitung
MODEL_NAME = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

def tokenize_fn(examples):
    return tokenizer(examples["text"], truncation=True, max_length=512)

tokenized_ds = dataset.map(tokenize_fn, batched=True,
                           remove_columns=["text"])
tokenized_ds = tokenized_ds.rename_column("label", "labels")
tokenized_ds.set_format("torch")
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

# 3. Modell laden
model = AutoModelForSequenceClassification.from_pretrained(
    MODEL_NAME, num_labels=2,
    id2label={0: "NEGATIVE", 1: "POSITIVE"},
    label2id={"NEGATIVE": 0, "POSITIVE": 1},
)

# 4. Evaluierungsmetriken
accuracy = evaluate.load("accuracy")
f1 = evaluate.load("f1")
def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    return {"accuracy": accuracy.compute(predictions=preds, references=labels)["accuracy"],
            "f1": f1.compute(predictions=preds, references=labels, average="binary")["f1"]}

# 5. Trainingsargumente
training_args = TrainingArguments(
    output_dir="./results", num_train_epochs=3,
    per_device_train_batch_size=16, per_device_eval_batch_size=32,
    gradient_accumulation_steps=2, learning_rate=2e-5,
    weight_decay=0.01, warmup_ratio=0.1,
    evaluation_strategy="steps", eval_steps=500,
    save_strategy="steps", save_steps=500,
    load_best_model_at_end=True, metric_for_best_model="f1",
    fp16=True, logging_steps=100, seed=42,
)

# 6. Trainer erstellen und trainieren
trainer = Trainer(
    model=model, args=training_args,
    train_dataset=tokenized_ds["train"],
    eval_dataset=tokenized_ds["test"],
    tokenizer=tokenizer, data_collator=data_collator,
    compute_metrics=compute_metrics,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=3)],
)
trainer.train()

# 7. Evaluieren und speichern
eval_result = trainer.evaluate()
print(f"Accuracy: {eval_result['eval_accuracy']:.4f}")
print(f"F1: {eval_result['eval_f1']:.4f}")

trainer.save_model("./my-sentiment-model")
tokenizer.save_pretrained("./my-sentiment-model")

LoRA parametereffizientes Fine-Tuning (empfohlen)

LoRA principle: only train low-rank matrices, freeze the original weights Full Fine-tuning W + ΔW Update all weight matrices Parameter count: all (e.g., train all 7B parameters) GPU memory requirement: extremely high (requires 4x model size) Cost: expensive, requires many GPUs LoRA fine-tuning W (frozen) Not trained + B x A r << d (low-rank matrix) Only train here Parameter count: 0.1% to 1% of the original GPU memory requirement: low (only stores low-rank matrix gradients) Cost: a single 24GB GPU can fine-tune a 7B model LoRA decomposes ΔW into two low-rank matrices, where r is much smaller than d (typically r=8~64), greatly reducing training cost.

Beispiel

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
import torch

model_name = "meta-llama/Llama-2-7b-hf"
model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.float16, device_map="auto",
    load_in_4bit=True,        # 4-Bit-quantisierte Ladung (QLoRA), spart zusätzlich GPU-Speicher
)

# LoRA konfigurieren
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM, r=16, lora_alpha=32,
    lora_dropout=0.05,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# -> trainierbar: 4,194,304 || alle: 6,742,609,920 || 0.0622%

LoRA-Vorteile: Es trainiert nur weniger als 1% der Parameter, reduziert den GPU-Speicher um 60-70%, ist 2-3x schneller, Gewichtsdateien sind nur wenige MB groß, und für dasselbe Basismodell können mehrere LoRA-Adapter für verschiedene Aufgaben gespeichert werden.

Speichern, Laden und Veröffentlichen von Modellen

Beispiel

# Lokal speichern
model.save_pretrained("./my-model")
tokenizer.save_pretrained("./my-model")

# Lokal laden
model = AutoModelForSequenceClassification.from_pretrained("./my-model")
tokenizer = AutoTokenizer.from_pretrained("./my-model")

# Auf HuggingFace Hub veröffentlichen
from huggingface_hub import login
login(token="your_hf_token")   # huggingface.co/settings/tokens
model.push_to_hub("your-username/my-sentiment-model")
tokenizer.push_to_hub("your-username/my-sentiment-model")

# Direkt über Trainer veröffentlichen
training_args = TrainingArguments(
    output_dir="your-username/my-model",
    push_to_hub=True, hub_strategy="every_save",
)

Tipps zur Leistungsoptimierung

Überblick über die Inferenzbeschleunigung

Inference Acceleration Tech Stack: From Simple to Deep Optimization Level 1 - Zero-Cost Optimization torch.no_grad() inference | fp16/bf16 half precision | batch inference | device_map="auto" automatic device assignment Speedup: 1.5~2x Level 2 - Quantization (Model Compression) bitsandbytes 4-bit/8-bit quantization | GPTQ (post-training quantization) | AWQ (activation-aware quantization) Speedup: 2~4x, VRAM halved Level 3 - Compilation and Runtime Optimization torch.compile()（PyTorch 2.0）| FlashAttention-2 | xFormers | Optimum（TensorRT/ONNX） Speedup: 3~10x Level 4 - Specialized Inference Engines vLLM (LLM high-throughput inference) | TGI | TensorRT-LLM | llama.cpp (CPU) Production-grade, highest performance

Beispiel

# 4-Bit-quantisiertes Laden (13B-Modell benötigt nur ~7GB VRAM)
from transformers import BitsAndBytesConfig, AutoModelForCausalLM
import torch

quant_config = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4",
)
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-13b-hf",
    quantization_config=quant_config, device_map="auto",
)

# FlashAttention-2-Beschleunigung (erfordert pip install flash-attn)
model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    attn_implementation="flash_attention_2",
    torch_dtype=torch.bfloat16, device_map="auto",
)

# torch.compile (PyTorch 2.0+)
model = torch.compile(model, mode="reduce-overhead")

Fehlerbehebung bei häufigen Problemen

Quick Reference for Common Errors and Solutions Error Message Cause and Solution CUDA out of memory Out of VRAM Reduce batch_size; add gradient_accumulation_steps; use fp16/4-bit; use a smaller model OSError: model not found Model name error or network issue Check spelling; set HF_ENDPOINT mirror; use local path if already downloaded ValueError: num_labels mismatch Explicitly specify num_labels=your number of classes when loading the model tensors on different devices inputs = {k: v.to(model.device) for k, v in inputs.items()} loss = NaN / loss not decreasing Check learning rate (too large causes NaN); check labels range (0~N-1); add gradient clipping slow tokenizer / slow speed pip install tokenizers to install the fast Rust version; use use_fast=True (default)

Zusammenfassung und Lernpfad

Transformers Learning Path and Core Knowledge Points Phase 1 Getting Started (1 week) pip install and Environment Setup Run 5 Tasks with Pipeline Understand the Tokenizer Workflow Manual Inference with AutoModel Understand Encoder/Decoder Understand Model Output Structure Phase 2 Advanced (2 weeks) Load Models from the Hub Hands-on Fine-tuning for Text Classification Master the Trainer API Custom Dataset Processing Evaluation with compute_metrics Save and Publish to the Hub Phase 3 Advanced (3 weeks) LoRA / PEFT Fine-tuning Quantization (4-bit/8-bit) Using Multimodal Models Custom Training Loop Accelerate Multi-GPU FlashAttention Acceleration Phase 4 Expert (Ongoing) Custom Model Architecture Pretraining from Scratch vLLM Production Deployment RLHF/DPO Alignment Contributing Open-Source Models Research Cutting-Edge Papers Each phase should include hands-on practice: find a real dataset and run through the complete workflow of training -> evaluation -> publishing

Kurzreferenz der wichtigsten API

Beispiel

# 1. Tokenizer laden
tokenizer = AutoTokenizer.from_pretrained("model_name")

# 2. Text kodieren
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

# 3. Modell laden
model = AutoModelForSequenceClassification.from_pretrained("model_name", num_labels=N)

# 4. Inferenz
with torch.no_grad():
    outputs = model(**inputs)

# 5. Pipeline
pipe = pipeline("task_name", model="model_name")

# 6. Trainingskonfiguration
args = TrainingArguments(output_dir="./out", num_train_epochs=3, learning_rate=2e-5)

# 7. Training
trainer = Trainer(model=model, args=args, train_dataset=ds, compute_metrics=fn)

# 8. Speichern
model.save_pretrained("./my-model")
tokenizer.save_pretrained("./my-model")

# 9. Datensatz
dataset = load_dataset("dataset_name")
dataset = dataset.map(tokenize_fn, batched=True)

# 10. LoRA
config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj","v_proj"])
model = get_peft_model(model, config)

Referenzressourcen

Andere Erweiterungen