I’ve always been a pip
+ venv
(yes, not even virtualenv
) user. Conda never played nice with me. However, there have been some times when I just could not install a requirements .txt
from a repository due to it always crashing with some dependency mismatches. Recently, I have heard more and more people mention how they use Poetry for their virtual environments and package management, so I decided to give it a try.
Since I’m mentioning it in a few of my blog posts now, I thought I would quickly write down how I currently use it – specifically I’m developing on macOS, so some instructions might be targeted towards that OS.
Table of Contents
- Installing poetry
- Creating a new project and virtual environment.
- The virtual environment & interpreter
- Installing new packages with poetry
- Running Scripts
- Activating the virtual environment with poetry manually
- Further Reading
Installing poetry
First off, you need Python 3.8+ to use this.
The official poetry documentation then recommends installing it via pipx
, which I had admittedly never used before either.
Pipx is installed either through brew
on macOS or scoop
on Windows, though there are some alternative options, which you can check on the official install page. Here’s how that looks on Mac:
brew install pipx
pipx ensurepath
sudo pipx ensurepath --global # optional to allow pipx actions in global scope
Then you can simply install poetry using
pipx install poetry
Creating a new project and virtual environment.
The following creates a new folder in the current directory with the given name and this new folder will be a poetry project:
poetry new llm-langfuse-demo
You can also initiate a poetry project in an already existing directory with
poetry init
In both cases, poetry creates a virtual environment for that project. By default, this environment is saved under {cache-dir}/virtualenvs
, with cache-dir=~/Library/Caches/pypoetry
as a default on macOS but you can change that.
The virtual environment & interpreter
For many use cases, like adding new packages or running scripts, you don’t need to know where the virtual environment files are saved. Poetry automatically manages the environment as we will see below. However, there are a few exceptions:
Setting the poetry environment as Python interpreter in your VSCode project
VSCode checks your code for errors and notifies you with warnings if you don’t have specific packages installed that you import in your scripts. To keep this information accurate, you need to tell it which environment and Python interpreter you are currently using.
To fix this, use the following terminal command to copy the path to the interpreter:
poetry env info --path | pbcopy
Then press Cmd + Shift + P to open the prompt for “Python > Set Interpreter”, then click “Enter interpreter path…” then paste.
Problem solved 😊 I haven’t yet found a way to do this automatically, but at least this is fast.
Source: VSCode: Adding Poetry Python Interpreter | Mark Needham
Installing new packages with poetry
While inside the folder of your project, simply running
poetry add <PyPI package>
adds the package as a dependency to your project (kind of like adding it to the requirements.txt
) and also installs the package into the virtual environment of your current directory.
You can check which virtual environment is currently being used by running:
poetry env info
You can also modify the pyproject.toml
file (sort of the replacement for requirements.txt
) by hand and add your packages there.
Running Scripts
To run a Python script within the created project, use
poetry run python your_script.py
If executed in the project directory created through poetry new ...
or poetry init
, this will use the underlying virtual environment without you needing to activate it explicitly. One less thing to remember, though it also can feel spooky if you are used to manually activating it.
Activating the virtual environment with poetry manually
Although the above commands already cover most use cases I encounter in my daily coding and deal with the underlying environment automatically, you can also activate the virtual environment explicitly by creating a nested shell with poetry shell
.
You deactivate the environment by simply exiting this shell with exit
.
This might be useful if you have to run scripts again and again and are tired of typing poetry run
or if your IDE automatically detects activated environments.
Further Reading
Guide on why you should use virtual environments by Princeton: https://csguide.cs.pr
Official Docs from Poetry: https://python-poetry.org/docs/