While working with Python’s machine learning and artificial intelligence packages, you may encounter an error as follows:
TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
This error occurs because the
protobuf library introduced breaking changes in version 4.21.0, specifically:
Python upb requires generated code that has been generated from protoc 3.19.0 or newer.
The following tutorial shows why this error occurs and ways you can fix it.
Why TypeError: Descriptors cannot not be created directly occurs
This error may happen when you install or run any Python code that uses the
protobuf package is a dependency for many machine learning and AI packages, so the package can be installed without your command.
For example, the
azureml libraries all has the
protobuf package as a dependency.
If you use any one of these libraries, then you need to fix the error in one of two ways:
protobufto version 3.20.x
- If your ML and AI libraries have fixed the error, upgrade them to the latest version
- Export the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python variable
Let’s see how to fix this error next.
How to fix TypeError: Descriptors cannot not be created directly
To easiest way to fix this error is to downgrade the
protobuf library installed on your Python system to version 3.20.x
Use one of the following commands to install
protobuf version 3.20.x:
pip install 'protobuf==3.20.*' # For pip3: pip3 install 'protobuf==3.20.*'
Once done, you can check the
protobuf package version by running the
pip show command as follows:
pip show protobuf #or pip3 show protobuf
Notice the version number in the output below:
Name: protobuf Version: 3.20.3 Summary: Protocol Buffers Home-page: https://developers.google.com/protocol-buffers/ Author: Author-email: License: BSD-3-Clause Location: /opt/homebrew/lib/python3.10/site-packages Requires: Required-by: tensorboard
If you have a
requirements.txt file, you can specify a dependency to
protobuf version 3.20.x as follows:
###### requirements.txt ###### protobuf==3.20.*
This way, the
protobuf version should be compatible with your ML libraries.
Another way to fix this error is to upgrade your ML libraries.
For example, TensorFlow version 2.7.3 fixed this error by specifying the max version range for the
Upgrade the library with one of the following commands:
pip install tensorflow --upgrade # or: pip3 install tensorflow --upgrade
Other libraries might have a similar fix released, but you need to check their websites for more information.
The third solution to this error is to export the
You need to run the following command from the terminal:
# Linux / macOS export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python # Windows Command Prompt setx PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION python # PowerShell $Env:PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="python"
If you’re deploying to a cloud-based service such as Azure or Google Cloud, then find out how to add an environment variable to your deployment to add the
But keep in mind that this solution uses a pure Python implementation for protocol parsing, so processing your code might be slower.
TypeError: Descriptors cannot not be created directly is caused by a breaking change in
protobuf version 4.
To fix this error, you can downgrade
protobuf to version 3.20.x or set the environment variable to
Some libraries that depend on
protobuf have addressed this error by adding the upper bound spec for
protobuf to only install version 3.20.x.
I hope this tutorial was helpful. Until next time! 🙏