<- prev

Running My First Local LLM on My MacBook Air (2026-09-13)

I use MacBook Air 2024 (Apple M3).


$ python3 --version
Python 3.14.7

$ python3 -m venv foo

$ source foo/bin/activate

(foo) $ pip install transformers torch accelerate

Source code: chat.py


import warnings

# Import Hugging Face Hub logging utility
from huggingface_hub.utils import logging as hf_logging

# Import Transformers model architecture and pipeline modules
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, logging as tf_logging

# Suppress warnings and logs
warnings.filterwarnings("ignore")
hf_logging.set_verbosity_error()
tf_logging.set_verbosity_error()

print("Download from Hugging Face Hub and load tokenizer")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

print("Download from Hugging Face Hub and load model")
model = AutoModelForCausalLM.from_pretrained(
  "microsoft/Phi-3-mini-4k-instruct",
  device_map="auto",
  torch_dtype="auto",
  trust_remote_code=False,
)

print("Create a pipeline")
generator = pipeline(
  "text-generation",
  model=model,
  tokenizer=tokenizer,
  return_full_text=False,
  max_new_tokens=300,
  do_sample=False,
  clean_up_tokenization_spaces=False,
)

print("\n--- Chat session started (Type 'exit' or 'quit' to end) ---\n")

# Start interactive loop with exception handling
try:
  while True:
    user_input = input("Prompt: ")
    
    # Check for exit command
    if user_input.strip().lower() in ["exit", "quit"]:
      print("Goodbye!")
      break
    
    # Skip empty input
    if not user_input.strip():
      continue

    # Show processing status
    print("\nThinking...", flush=True)

    output = generator(user_input)
    print("\nOutput:\n" + output[0]['generated_text'] + "\n")

except KeyboardInterrupt:
  print("\nGoodbye!")

Run it.


(foo) $ python chat.py

Running My First Local LLM on My MacBook Air

Posted by NI-Lab. (@nilab)

<- prev