Sqids – Generate Short Unique IDs from Numbers
3 days, 15 hours ago

Article URL: https://sqids.org/

Comments URL: https://news.ycombinator.com/item?id=38414914

Points: 105

# Comments: 31

Show HN: Personalized Meal Plans to Suit Your Lifestyle
3 days, 17 hours ago

Hey HN, I'm Patrick a developer working on side projects beside a 9-5. I created a Site where you can create customised Meal Plans + Grocery Lists and get them automatically every Week into your Inbox.

Please Check it out, any feedback is welcome!


Comments URL: https://news.ycombinator.com/item?id=38413970

Points: 11

# Comments: 6

Show HN: I saw this mind-blowing experiment, so I made a simple version of it
3 days, 18 hours ago

Two browser windows (acting as socket clients) communicate their:

- Screen dimensions - (screen.width, screen.height)

- Window dimensions - (window.innerWidth, window.innerHeight)

- Window X/Y position - (window.screenX, window.screenY)

...or whichever calculation works best for you.

The original work by Bjorn Staal https://twitter.com/_nonfigurativ_/status/172732259457002734 used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

Here's a demo of how it works and the codebase: https://github.com/Momciloo/fun-with-sockets/

Comments URL: https://news.ycombinator.com/item?id=38413660

Points: 100

# Comments: 35

Show HN: We made an interactive coloring experience for kids
3 days, 19 hours ago

Hi HN community!

Proud to have recently become a founder myself and now to submit a "Show HN".

We are Dibulo, a young startup which is on a mission to redefine coloring sessions for kids! We have built our first worlds, sold our first coloring books and now we want to let the world know about us :) So far we have been praised by parents, and also got pretty good feedback from institutions like daycares and elementary school teachers.

What sets our coloring experience apart? Not only do we bring vibrant templates on paper to life on screens, but we also introduce a captivating group activity. Picture this: kids enthusiastically coloring away, their creations showcased prominently on a big screen for all to admire.

For us it is crucial that the user experience is smooth and easy. Our platform works in pretty much all modern browsers and on all major devices (PC, laptop, tablet, AndroidTV / FireTV). Setting up a coloring session on Dibulo takes less time than a kid can color our templates and uploading is as easy as snapping a picture. You can even share the coloring session with your family and friends and if they like they can participate and upload their drawings too (remotely).

Feel free to download and print our free templates and try them with your kids / nephews / at your daycare.

Thank you for checking us out! Looking forward to receiving feedback!

Comments URL: https://news.ycombinator.com/item?id=38413520

Points: 10

# Comments: 3

Show HN: IP Guide – info on IP addresses, Networks, and ASNs
4 days, 10 hours ago

Hey all, wanted to share my recent weekend hack project. I wanted to improve upon the existing space for looking up information about IP addresses and ASNs.

The backend is written in Rust and pulls BGP data every 8 hours from the RIPE RIS project to build up a routing table and also adds geolocation information. The frontend is using React and Tailwind.

Would love any feedback or suggestions on what to improve.

Comments URL: https://news.ycombinator.com/item?id=38409164

Points: 108

# Comments: 38

Julia as a unifying end-to-end workflow language on the Frontier exascale system (Sep. 2023)
1 week, 1 day ago

By William F. Godoy, Pedro Valero-Lara, Caira Anderson, Katrina W. Lee, Ana Gainaru, Rafael Ferreira da Silva, Jeffrey S. Vetter.


We evaluate Julia as a single language and ecosystem paradigm powered by LLVM to develop workflow components for high-performance computing. We run a Gray-Scott, 2-variable diffusion-reaction application using a memory-bound, 7-point stencil kernel on Frontier, the US Department of Energy’s first exascale supercomputer. We evaluate the performance, scaling, and trade-offs of (i) the computational kernel on AMD’s MI250x GPUs, (ii) weak scaling up to 4,096 MPI processes/GPUs or 512 nodes, (iii) parallel I/O writes using the ADIOS2 library bindings, and (iv) Jupyter Notebooks for interactive analysis. Results suggest that although Julia generates a reasonable LLVM-IR, a nearly 50% performance difference exists vs. native AMD HIP stencil codes when running on the GPUs. As expected, we observed near-zero overhead when using MPI and parallel I/O bindings for system-wide installed implementations. Consequently, Julia emerges as a compelling high-performance and high-productivity workflow composition language, as measured on the fastest supercomputer in the world.


Typst – Compose Papers Faster
1 week, 1 day ago

Article URL: https://typst.app/

Comments URL: https://news.ycombinator.com/item?id=38354422

Points: 124

# Comments: 36

DOS/4GW and Protected Mode
1 week, 1 day ago
What are you doing this week?
1 week, 1 day ago

What are you doing this week? Feel free to share!

Keep in mind it’s OK to do nothing at all, too.

Victor Mono Typeface
1 week, 2 days ago

Article URL: https://rubjo.github.io/victor-mono/

Comments URL: https://news.ycombinator.com/item?id=38332635

Points: 101

# Comments: 73

🎉 PyTorch Docathon H2 2023 Wrap-up 🎉
1 week, 6 days ago
We are thrilled to announce the successful completion of the Fall 2023 PyTorch Docathon! The event was a resounding success, and we want to extend our heartfelt gratitude to all the participants who made it possible. Dedication, expertise, and tireless efforts of our open-source contributors have once again helped us to improve PyTorch documentation.
Accelerating Generative AI with PyTorch: Segment Anything, Fast
1 week, 6 days ago
This post is the first part of a multi-series blog focused on how to accelerate generative AI models with pure, native PyTorch. We are excited to share a breadth of newly released PyTorch performance features alongside practical examples of how these features can be combined to see how far we can push PyTorch native performance.
1 week, 6 days ago

Robust Speech Recognition via Large-Scale Weak Supervision


[Blog] [Paper] [Model card] [Colab example]

Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.


A Transformer sequence-to-sequence model is trained on various speech processing tasks, including multilingual speech recognition, speech translation, spoken language identification, and voice activity detection. These tasks are jointly represented as a sequence of tokens to be predicted by the decoder, allowing a single model to replace many stages of a traditional speech-processing pipeline. The multitask training format uses a set of special tokens that serve as task specifiers or classification targets.


We used Python 3.9.9 and PyTorch 1.10.1 to train and test our models, but the codebase is expected to be compatible with Python 3.8-3.11 and recent PyTorch versions. The codebase also depends on a few Python packages, most notably OpenAI's tiktoken for their fast tokenizer implementation. You can download and install (or update to) the latest release of Whisper with the following command:

pip install -U openai-whisper

Alternatively, the following command will pull and install the latest commit from this repository, along with its Python dependencies:

pip install git+https://github.com/openai/whisper.git 

To update the package to the latest version of this repository, please run:

pip install --upgrade --no-deps --force-reinstall git+https://github.com/openai/whisper.git

It also requires the command-line tool ffmpeg to be installed on your system, which is available from most package managers:

# on Ubuntu or Debian
sudo apt update && sudo apt install ffmpeg

# on Arch Linux
sudo pacman -S ffmpeg

# on MacOS using Homebrew (https://brew.sh/)
brew install ffmpeg

# on Windows using Chocolatey (https://chocolatey.org/)
choco install ffmpeg

# on Windows using Scoop (https://scoop.sh/)
scoop install ffmpeg

You may need rust installed as well, in case tiktoken does not provide a pre-built wheel for your platform. If you see installation errors during the pip install command above, please follow the Getting started page to install Rust development environment. Additionally, you may need to configure the PATH environment variable, e.g. export PATH="$HOME/.cargo/bin:$PATH". If the installation fails with No module named 'setuptools_rust', you need to install setuptools_rust, e.g. by running:

pip install setuptools-rust

Available models and languages

There are five model sizes, four with English-only versions, offering speed and accuracy tradeoffs. Below are the names of the available models and their approximate memory requirements and inference speed relative to the large model; actual speed may vary depending on many factors including the available hardware.

Size Parameters English-only model Multilingual model Required VRAM Relative speed
tiny 39 M tiny.en tiny ~1 GB ~32x
base 74 M base.en base ~1 GB ~16x
small 244 M small.en small ~2 GB ~6x
medium 769 M medium.en medium ~5 GB ~2x
large 1550 M N/A large ~10 GB 1x

The .en models for English-only applications tend to perform better, especially for the tiny.en and base.en models. We observed that the difference becomes less significant for the small.en and medium.en models.

Whisper's performance varies widely depending on the language. The figure below shows a performance breakdown of large-v3 and large-v2 models by language, using WERs (word error rates) or CER (character error rates, shown in Italic) evaluated on the Common Voice 15 and Fleurs datasets. Additional WER/CER metrics corresponding to the other models and datasets can be found in Appendix D.1, D.2, and D.4 of the paper, as well as the BLEU (Bilingual Evaluation Understudy) scores for translation in Appendix D.3.

Command-line usage

The following command will transcribe speech in audio files, using the medium model:

whisper audio.flac audio.mp3 audio.wav --model medium

The default setting (which selects the small model) works well for transcribing English. To transcribe an audio file containing non-English speech, you can specify the language using the --language option:

whisper japanese.wav --language Japanese

Adding --task translate will translate the speech into English:

whisper japanese.wav --language Japanese --task translate

Run the following to view all available options:

whisper --help

See tokenizer.py for the list of all available languages.

Python usage

Transcription can also be performed within Python:

import whisper

model = whisper.load_model("base")
result = model.transcribe("audio.mp3")

Internally, the transcribe() method reads the entire file and processes the audio with a sliding 30-second window, performing autoregressive sequence-to-sequence predictions on each window.

Below is an example usage of whisper.detect_language() and whisper.decode() which provide lower-level access to the model.

import whisper

model = whisper.load_model("base")

# load audio and pad/trim it to fit 30 seconds
audio = whisper.load_audio("audio.mp3")
audio = whisper.pad_or_trim(audio)

# make log-Mel spectrogram and move to the same device as the model
mel = whisper.log_mel_spectrogram(audio).to(model.device)

# detect the spoken language
_, probs = model.detect_language(mel)
print(f"Detected language: {max(probs, key=probs.get)}")

# decode the audio
options = whisper.DecodingOptions()
result = whisper.decode(model, mel, options)

# print the recognized text

More examples

Please use the 🙌 Show and tell category in Discussions for sharing more example usages of Whisper and third-party extensions such as web demos, integrations with other tools, ports for different platforms, etc.


Whisper's code and model weights are released under the MIT License. See LICENSE for further details.

Embeddings and Vector Databases With ChromaDB
1 week, 6 days ago
Vector databases are a crucial component of many NLP applications. This tutorial will give you hands-on experience with ChromaDB, an open-source vector database that's quickly gaining traction. Along the way, you'll learn what's needed to understand vector databases with practical examples.
Python Basics: Modules and Packages
2 weeks ago
In this Python Basics video course, you'll learn how to build an application by putting related code into separate files called modules. You'll also use the import statement to use modules in another file.
JupyterLab for an Enhanced Notebook Experience
2 weeks, 1 day ago
In this tutorial, you'll learn how to use the JupyterLab authoring environment and what it brings to the popular computational notebook Jupyter Notebook. You'll learn about its different tools and discover how they can work together to enhance your notebook experience.
Level Up Your Python
2 weeks, 3 days ago
2 weeks ago

🚀🧠💬 Supercharged Custom Instructions for ChatGPT (non-coding) and ChatGPT Advanced Data Analysis (coding).

ChatGPT AutoExpert

by Dustin Miller • RedditSubstack

License: Attribution-NonCommercial-ShareAlike 4.0 International

Elevating Conversational AI to Expert Level

❇️ NEW ❇️

I've created a set of "Custom GPTs" with updated versions of these prompts:

Want to support these free prompts? My Substack offers paid subscriptions, that's the best way to show your appreciation.


ChatGPT AutoExpert is a shockingly effective set of custom instructions aimed at enhancing the capabilities of GPT-4 and GPT-3.5-Turbo conversational models. These instructions maximize the depth and nuance in responses while minimizing general disclaimers and hand-holding. The ultimate objective is to provide users with accurate, context-rich information and an improved learning experience.

Getting Started

To get started with ChatGPT AutoExpert, choose which set of custom instructions you want to use:


"Standard Edition"

  • ✳️ New to v5: Automatically Improves your Question
    Many of us still compose ambiguous questions when asking ChatGPT for help. The AutoExpert will automatically rewrite your question to be precise, and to elicit the best response the experts can provide.
  • ✳️ New to v5: Slash Commands
    Slash commands offer an easy way to interact with the AutoExpert system. Get summaries, ideas for additional questions, alternative viewpoints…even ask ChatGPT to review its own answer and suggest improvements.
  • ✳️ New to v5: Auto-selects Frameworks and Methodologies
    Designed to select a context-appropriate framework for formulating its best answers
  • Maximized Depth and Nuance
    Receive high-quality, in-depth, and ✳️ New to v5: multi-turn responses (GPT-4 only) without compromising on the granularity of the information.
  • Perfect for Everyday Use
    No need to switch these instructions on and off. They'll give you a greatly improved experience with ChatGPT, even if you're writing code. Although, if you are writing code, you should check the Developer Edition
  • Automatically Identifies the Best Expert
    The AutoExpert custom instruction automatically finds the best expert roles to answer whatever question you ask, every time. You don't need a bunch of special prompts any more—this works with even the simplest of prompts!
  • Minimized Hand-Holding
    Cut through the noise and get straight to the facts, reducing unnecessary disclaimers.
  • Explicit Reasoning
    Your AutoExpert doesn't just provide answers; it offers an explanation, detailing the thought process behind each response.
  • Resourceful Links
    Automatically generates inline links for related topics and "you may also like" topics, helpfully linked to Google search results to avoid hallucination (GPT-3.5 still hallucinates here, but not always. GPT-4 is rock-solid).

"Developer Edition"

[!IMPORTANT] This requires a ChatGPT professional subscription, as it needs both GPT-4 and Advanced Data Analysis!

  • Verbosity Selection
    Easily choose the complexity of the generated code, from compact "code golf" type responses, up to complex, modular code samples
  • Powered by Jupyter
    ChatGPT Advanced Data Analysis already runs a Jupyter kernel under the hood. AutoExpert (Developer Edition) comes with a companion Python script that you simply upload to your conversation. It will automatically take advantage of the sandbox Python environment for editing longer code samples, and activate a handful of extra "slash commands" to make your life even easier.
  • Pick Up Where You Left Off
    You can start a new chat without worrying about ChatGPT forgetting what you were doing in the previous one. The /memory slash command will download all your files, and a history of everything that's been done during your session. Simply upload it (along with the companion script) in a new session, and pick up where you left off.
  • Install Custom Wheels
    Yeah, you heard me. Wheels for Python packages can be uploaded and installed automatically.
    • Note that your ChatGPT sandbox uses Python 3.8, on a VM with x86_64 architecture (as of this writing).
  • Save Your Work
    Among other /slash commands, AutoExpert (Developer Edition) will save all your code snippets, dehydrate its memory of your requirements and the work it's done—even back up the code cells themselves. Then it zips it up, and you can quickly download your coding conversation history.
  • File and Symbol Tree
    By keeping a running history along with a file/symbol tree at the end of each response, ChatGPT will always remember what it just did, and you'll always see what files still need work. It's even smart enough to handle breaking down complex requirements in a way that allows it to write code over multiple turns.

ChatGPT AutoExpert (both standard and "Developer Edition")
by Dustin Miller is licensed under Attribution-NonCommercial-ShareAlike 4.0 International

2 weeks, 1 day ago

uBlock Origin - An efficient blocker for Chromium and Firefox. Fast and lean.

uBlock Origin (uBO)

BEWARE! uBO is (and has always been) COMPLETELY UNRELATED to the website ublock.org.

uBlock Origin (uBO) is a CPU and memory-efficient wide-spectrum content blocker for Chromium and Firefox. It blocks ads, trackers, coin miners, popups, annoying anti-blockers, malware sites, etc., by default using EasyList, EasyPrivacy, Peter Lowe's Blocklist, Online Malicious URL Blocklist, and uBO filter lists. There are many other lists available to block even more. Hosts files are also supported. uBO uses the EasyList filter syntax and extends the syntax to work with custom rules and filters.

You may easily unselect any preselected filter lists if you think uBO blocks too much. For reference, Adblock Plus installs with only EasyList, ABP filters, and Acceptable Ads enabled by default.

It is important to note that using a blocker is NOT theft. Do not fall for this creepy idea. The ultimate logical consequence of blocking = theft is the criminalization of the inalienable right to privacy.

Ads, "unintrusive" or not, are just the visible portion of the privacy-invading means entering your browser when you visit most sites. uBO's primary goal is to help users neutralize these privacy-invading methods in a way that welcomes those users who do not wish to use more technical means.


Basic Mode Advanced Mode
The simple popup user interface for an install-it-and-forget-it type of installation that is configured optimally by default. The advanced popup user interface includes a point-and-click firewall that is configurable on a per-site basis.

Visit the Wiki for documentation.

For support, questions, or help, visit /r/uBlockOrigin.


Required Permissions


Firefox Add-ons

Development Builds

uBO works best on Firefox and is available for desktop and Android versions.


Thunderbird Add-ons

In Thunderbird, uBlock Origin does not affect emails, just feeds.


Chrome Web Store

Microsoft Edge Add-ons (Published by: Nicole Rolls)

Opera Add-ons

Development Builds

uBO should be compatible with any Chromium-based browser.

All Programs

Do NOT use uBO with any other content blocker. uBO performs as well as or better than most popular blockers. Other blockers can prevent uBO's privacy or anti-blocker-defusing features from working correctly.

Manual Installation

Enterprise Deployment

Deploying uBO

Release History

Releases Page


Help translate uBO via Crowdin.



Privacy Policy

GPLv3 License

Free. Open-source. For users by users. No donations sought.

If you ever want to contribute something, think about the people working hard to maintain the filter lists you are using, which are available to use by all for free.

2 weeks, 1 day ago

Playing Pokemon Red with Reinforcement Learning

Train RL agents to play Pokemon Red

Watch the Video on Youtube!

Join the discord server

Running the Pretrained Model Interactively 🎮

🐍 Python 3.10 is recommended. Other versions may work but have not been tested.
You also need to install ffmpeg and have it available in the command line.

  1. Copy your legally obtained Pokemon Red ROM into the base directory. You can find this using google, it should be 1MB. Rename it to PokemonRed.gb if it is not already. The sha1 sum should be ea9bcae617fdf159b045185467ae58b2e4a48b9a, which you can verify by running shasum PokemonRed.gb.
  2. Move into the baselines/ directory:
    cd baselines
  3. Install dependencies:
    pip install -r requirements.txt
    It may be necessary in some cases to separately install the SDL libraries.
  4. Run:
    python run_pretrained_interactive.py

Interact with the emulator using the arrow keys and the a and s keys (A and B buttons).
You can pause the AI's input during the game by editing agent_enabled.txt

Note: the Pokemon.gb file MUST be in the main directory and your current directory MUST be the baselines/ directory in order for this to work.

Training the Model 🏋️

10-21-23: Updated Version!

This version still needs some tuning, but it can clear the first gym in a small fraction of the time and compute resources. It can work with as few as 16 cores and ~20G of RAM. This is the place for active development and updates!

  1. Previous steps 1-3
  2. Run:
    python run_baseline_parallel_fast.py

Tracking Training Progress 📈

The current state of each game is rendered to images in the session directory.
You can track the progress in tensorboard by moving into the session directory and running:
tensorboard --logdir .
You can then navigate to localhost:6006 in your browser to view metrics.
To enable wandb integration, change use_wandb_logging in the training script to True.

Extra 🐜

Map visualization code can be found in visualization/ directory.

Supporting Libraries

Check out these awesome projects!


Stable Baselines 3

2 weeks, 2 days ago

Qt based cross-platform GUI proxy configuration manager (backend: v2ray / sing-box)

NekoRay / NekoBox For PC

Qt based cross-platform GUI proxy configuration manager (backend: v2ray / sing-box)

Support Windows / Linux out of the box now.

基于 Qt 的跨平台代理配置管理器 (后端 v2ray / sing-box)

目前支持 Windows / Linux 开箱即用

Readme Translations

中文 / English / fa_IR / 日本語

下载 / Download

GitHub Releases (Portable ZIP)

便携格式,无安装器。转到 Releases 下载预编译的二进制文件,解压后即可使用。

下载 / Download





Scoop Extras

scoop install nekoray

更改记录 & 发布频道 / Changelog & Telegram Channel


项目主页 & 文档 / Homepage & Documents


代理 / Proxy

  • SOCKS (4/4a/5)
  • HTTP(S)
  • Shadowsocks
  • VMess
  • Trojan
  • TUIC ( sing-box )
  • NaïveProxy ( Custom Core )
  • Hysteria ( Custom Core or sing-box )
  • Hysteria2 ( Custom Core or sing-box )
  • Custom Outbound
  • Custom Config
  • Custom Core

订阅 / Subscription

  • Raw: some widely used formats (like Shadowsocks, Clash and v2rayN)
  • 原始格式: 一些广泛使用的格式 (如 Shadowsocks、Clash 和 v2rayN)



Windows 运行

若提示 DLL 缺失,无法运行,请下载 安装 微软 C++ 运行库

Linux 运行

Linux 运行教程


由于缺乏维护,不再提供 macOS 版本下载。

您还可以在下面的存储库中非正式地获取更新版本。非官方发布 / Unofficial releases

仍可以参照下方教程自行编译,常见问题请参考历史 Issue,欢迎 Pull Request 修复。

编译教程 / Compile Tutorial

请看 技术文档 / Technical documentation

捐助 / Donate


捐赠满等额 50 USD 可以在「捐赠榜」显示头像,如果您未被添加到这里,欢迎联系我们补充。

Donations of 50 USD or more can display your avatar on the Donation List. If you are not added here, please contact us to add it.





  • 您也可以通过 Google Play 购买捐赠,另有支付宝微信购买方式,见 TG 频道置顶。
  • Donating via Google Play, using credit cards.




2 weeks, 5 days ago

A V2Ray client for Android, support Xray core and v2fly core


A V2Ray client for Android, support Xray core and v2fly core

Telegram Channel



Geoip and Geosite

  • geoip.dat and geosite.dat files are in Android/data/com.v2ray.ang/files/assets (path may differ on some Android device)
  • download feature will get enhanced version in this repo (Note it need a working proxy)
  • latest official domain list and ip list can be imported manually
  • possible to use third party dat file in the same folder, like h2y

More in our wiki

Development guide

Android project under V2rayNG folder can be compiled directly in Android Studio, or using Gradle wrapper. But the v2ray core inside the aar is (probably) outdated.
The aar can be compiled from the Golang project AndroidLibV2rayLite or AndroidLibXrayLite. For a quick start, read guide for Go Mobile and Makefiles for Go Developers

v2rayNG can run on Android Emulators. For WSA, VPN permission need to be granted via appops set [package name] ACTIVATE_VPN allow

2 weeks, 5 days ago

The cutest Discord client mod


The cutest Discord client mod


  • Super easy to install (Download Installer, open, click install button, done)
  • 100+ plugins built in: See a list
    • Some highlights: SpotifyControls, MessageLogger, Experiments, GameActivityToggle, Translate, NoTrack, QuickReply, Free Emotes/Stickers, PermissionsViewer, CustomCommands, ShowHiddenChannels, PronounDB
  • Fairly lightweight despite the many inbuilt plugins
  • Excellent Browser Support: Run Vencord in your Browser via extension or UserScript
  • Works on any Discord branch: Stable, Canary or PTB all work (though for the best experience I recommend stable!)
  • Custom CSS and Themes: Inbuilt css editor with support to import any css files (including BetterDiscord themes)
  • Privacy friendly, blocks Discord analytics & crash reporting out of the box and has no telemetry
  • Maintained very actively, broken plugins are usually fixed within 12 hours
  • Settings sync: Keep your plugins and their settings synchronised between devices / apps (optional)

Installing / Uninstalling

Visit https://vencord.dev/download

Join our Support/Community Server


Star History


Discord is trademark of Discord Inc. and solely mentioned for the sake of descriptivity. Mention of it does not imply any affiliation with or endorsement by Discord Inc.

Using Vencord violates Discord's terms of service

Client modifications are against Discord’s Terms of Service.

However, Discord is pretty indifferent about them and there are no known cases of users getting banned for using client mods! So you should generally be fine as long as you don’t use any plugins that implement abusive behaviour. But no worries, all inbuilt plugins are safe to use!

Regardless, if your account is very important to you and it getting disabled would be a disaster for you, you should probably not use any client mods (not exclusive to Vencord), just to be safe

Additionally, make sure not to post screenshots with Vencord in a server where you might get banned for it

2 weeks, 6 days ago

Langchain + Docker + Neo4j + Ollama

GenAI Stack

The GenAI Stack will get you started building your own GenAI application in no time. The demo applications can serve as inspiration or as a starting point. Learn more about the details in the technical blog post.


Create a .env file from the environment template file env.example

Available variables:

Variable Name Default value Description
OLLAMA_BASE_URL http://host.docker.internal:11434 REQUIRED - URL to Ollama LLM API
NEO4J_URI neo4j://database:7687 REQUIRED - URL to Neo4j database
NEO4J_USERNAME neo4j REQUIRED - Username for Neo4j database
NEO4J_PASSWORD password REQUIRED - Password for Neo4j database
LLM llama2 REQUIRED - Can be any Ollama model tag, or gpt-4 or gpt-3.5 or claudev2
EMBEDDING_MODEL sentence_transformer REQUIRED - Can be sentence_transformer, openai, aws or ollama
AWS_ACCESS_KEY_ID REQUIRED - Only if LLM=claudev2 or embedding_model=aws
AWS_SECRET_ACCESS_KEY REQUIRED - Only if LLM=claudev2 or embedding_model=aws
AWS_DEFAULT_REGION REQUIRED - Only if LLM=claudev2 or embedding_model=aws
OPENAI_API_KEY REQUIRED - Only if LLM=gpt-4 or LLM=gpt-3.5 or embedding_model=openai
LANGCHAIN_ENDPOINT "https://api.smith.langchain.com" OPTIONAL - URL to Langchain Smith API
LANGCHAIN_TRACING_V2 false OPTIONAL - Enable Langchain tracing v2
LANGCHAIN_PROJECT OPTIONAL - Langchain project name

LLM Configuration

MacOS and Linux users can use any LLM that's available via Ollama. Check the "tags" section under the model page you want to use on https://ollama.ai/library and write the tag for the value of the environment variable LLM= in th e.env file. All platforms can use GPT-3.5-turbo and GPT-4 (bring your own API keys for OpenAIs models).

MacOS Install Ollama on MacOS and start it before running docker compose up.

Linux No need to install Ollama manually, it will run in a container as part of the stack when running with the Linux profile: run docker compose --profile linux up. Make sure to set the OLLAMA_BASE_URL=http://llm:11434 in the .env file when using Ollama docker container.

To use the Linux-GPU profile: run docker compose --profile linux-gpu up. Also change OLLAMA_BASE_URL=http://llm-gpu:11434 in the .env file.

Windows Not supported by Ollama, so Windows users need to generate a OpenAI API key and configure the stack to use gpt-3.5 or gpt-4 in the .env file.


[!WARNING] There is a performance issue that impacts python applications in the 4.24.x releases of Docker Desktop. Please upgrade to the latest release before using this stack.

To start everything

docker compose up

If changes to build scripts has been made, rebuild.

docker compose up --build

To enter watch mode (auto rebuild on file changes). First start everything, then in new terminal:

docker compose watch

Shutdown If health check fails or containers don't start up as expected, shutdown completely to start up again.

docker compose down


Here's what's in this repo:

Name Main files Compose name URLs Description
Support Bot bot.py bot http://localhost:8501 Main usecase. Fullstack Python application.
Stack Overflow Loader loader.py loader http://localhost:8502 Load SO data into the database (create vector embeddings etc). Fullstack Python application.
PDF Reader pdf_bot.py pdf_bot http://localhost:8503 Read local PDF and ask it questions. Fullstack Python application.
Standalone Bot API api.py api http://localhost:8504 Standalone HTTP API streaming (SSE) + non-streaming endpoints. Python.
Standalone Bot UI front-end/ front-end http://localhost:8505 Standalone client that uses the Standalone Bot API to interact with the model. JavaScript (Svelte) front-end.

The database can be explored at http://localhost:7474.

App 1 - Support Agent Bot

UI: http://localhost:8501 DB client: http://localhost:7474

  • answer support question based on recent entries
  • provide summarized answers with sources
  • demonstrate difference between
    • RAG Disabled (pure LLM response)
    • RAG Enabled (vector + knowledge graph context)
  • allow to generate a high quality support ticket for the current conversation based on the style of highly rated questions in the database.

(Chat input + RAG mode selector)

(CTA to auto generate support ticket draft) (UI of the auto generated support ticket draft)

App 2 - Loader

UI: http://localhost:8502 DB client: http://localhost:7474

  • import recent Stack Overflow data for certain tags into a KG
  • embed questions and answers and store in vector index
  • UI: choose tags, run import, see progress, some stats of data in the database
  • Load high ranked questions (regardless of tags) to support the ticket generation feature of App 1.

App 3 Question / Answer with a local PDF

UI: http://localhost:8503
DB client: http://localhost:7474

This application lets you load a local PDF into text chunks and embed it into Neo4j so you can ask questions about its contents and have the LLM answer them using vector similarity search.

App 4 Standalone HTTP API


Example cURL command:

curl http://localhost:8504/query-stream\?text\=minimal%20hello%20world%20in%20python\&rag\=false

Exposes the functionality to answer questions in the same way as App 1 above. Uses same code and prompts.

App 5 Static front-end

UI: http://localhost:8505

This application has the same features as App 1, but is built separate from the back-end code using modern best practices (Vite, Svelte, Tailwind).
The auto-reload on changes are instant using the Docker watch sync config.

3 weeks ago

Xray, Penetrates Everything. Also the best v2ray-core, with XTLS support. Fully compatible configuration.

Project X

Project X originates from XTLS protocol, providing a set of network tools such as Xray-core and REALITY.

README is open, so feel free to submit your project here.


Mozilla Public License Version 2.0


Project X Official Website


Project X

Project X Channel



GUI Clients

Others that support VLESS, XTLS, REALITY, XUDP, PLUX...


Code of Conduct




go build -o xray.exe -trimpath -ldflags "-s -w -buildid=" ./main

Linux / macOS

go build -o xray -trimpath -ldflags "-s -w -buildid=" ./main

Stargazers over time

Join two models on geometry predicate
2 weeks ago

I’d like to know if anyone already succeeded in joining two models that relate only on their geometry fields, without raw SQL. My use case is to annotate a parent model containing a polygon field with the number of points (PointField of another model) contained in each parent model instance.

I found this StackOverflow post (python - GeoDjango query: all point that are contained into a multi polygon - Stack Overflow) which is close to what I’m trying to achieve, but I’m unfortunately getting errors. I can develop my trials/errors, but I’d like first to know if someone found a solution for a similar use case.

Any success story here?

1 post - 1 participant

Read full topic

Build a Reactive ToDo application in Django with Django-Unicorn
2 weeks ago

In my latest blog post, “Build a Reactive ToDo Application in Django with Django-Unicorn,” Idelve into the world of efficient web application development using the robust combination of Django and Django-Unicorn.

This comprehensive guide is designed for developers looking to create a ToDo application that is not only functional but also highly reactive, providing a smooth, real-time user experience.

With Django-Unicorn, I demonstrate how to bridge the gap between the simplicity of Django and the reactivity of modern JavaScript frameworks, enabling you to write less JavaScript and more Python.

Build a Reactive ToDo application in Django with Django-Unicorn

1 post - 1 participant

Read full topic

Ticket #21183 reopening?
2 weeks, 2 days ago

I’d be interested in reopening ticket #21183 (Filter block on the right covers information in admin) – Django . This ticket was in limbo for very long time before going from “wontfix” → “new” → “closed” recently with a rationale that another ticket is implemented (#27471 (Make admin's list_filter choices collapsable) – Django) and “we dont need both of them”.

However the latter ticket is about collapsing individual items in the sidebar vertically, whereas the former one is about hiding the whole sidebar which provides more horizontal screen space, so the only similarity here is that these are related to the same sidebar (?)

The commenter reopening given ticket states that they’re affected by this, so implementation of #27471 havent apparently solved their issue.
I’m facing the same problem, so I’d like to challenge the decision for having this ticket closed, and I’m willing to contribute the code in case it could be reopened.

Thanks for consideration.

5 posts - 3 participants

Read full topic

Multi-tenancy on Django via Postgres row level security
2 weeks, 2 days ago

Hey everyone. After spending some years using and promoting Postgres schemas for multi-tenancy in Django, I decided to give row level security a try.


GitHub - lorinkoz/django-pgrls: Django multi-tenancy through Postgres row...

Django multi-tenancy through Postgres row level security - GitHub - lorinkoz/django-pgrls: Django multi-tenancy through Postgres row level security

1 post - 1 participant

Read full topic

Change reusable apps naming recommendation
2 weeks, 2 days ago

The docs page Advanced tutorial: How to write reusable apps covers packaging a Django app for PyPI. The “Choosing a name for your app” admonition recommends using an example module name polls with a package name django-polls.

I propose changing this page to recommend a module name of django_polls and making the admonition warn against not using the django_ prefix. The current approach causes name clashes which prevents using packages together. Here are the examples I know of:

I’m sure there are more.

3 posts - 3 participants

Read full topic

Django active developers Sprints proposal
2 weeks, 4 days ago

A year ago, during DjangoCon US 2022, I proposed the idea of organizing the “Django active developers Sprints”

Since then, I’ve talked about it with some community members and I updated the article.

I’ll share it again

1 post - 1 participant

Read full topic

Checklists for upgrading packages and upgrading Python - Public Notes
2 weeks, 4 days ago

In the spirit of creating public notes (which I heard about from Ryan Cheley during his talk at DjangoCon US 2023), I’m posting on the chance it could be helpful for anyone else who is also learning.

Feedback welcome - as with everything, I’m sure there are iterations to be made as I learn more (just please keep in mind that I’m a beginner if responding).

Upgrading packages checklist [check ~monthly]

  • Check for deprecation warnings by running the project’s tests
    python -Wa manage.py test >>> use if using Django Test module
    PYTHONWARNINGS=always pytest --capture=no >>> use if using pytest

    • For each deprecation warning, address what it’s warning about & then rerun the tests
  • Check if there are any outdated dependencies / package versions in the project:
    pip list --outdated

  • For each outdated package,

    • In requirements.in (or dev-requirements.in) file, update the version number of OutdatedPackageName to upgrade to the next incremental (e.g. from X.Y.Z to X.Y+1.Z to X.Y+2.X) release
      Note: OutdatedPackageName~=2.1.0 will install the latest version of OutdatedPackageName 2.1.X where X will update automatically when syncing your venv if a newer version is available (aka you won’t manually need to type the 2.1.1 to 2.1.2 to 2.1.3 updates in the requirements.in file each time)
      Note: Search for “OutdatedPackageName X.X.X Release Notes” to find info about changes in that version. Likely will also be in OutdatedPackageName GitHub Repo
  • Save file

  • Compile updated requirements.txt or (dev-requirements.txt)
    pip-compile requirements.in
    pip-compile dev-requirements.in

  • Sync your venv to make it match your dev-requirements.txt file
    pip-sync dev-requirements.txt
    Note: you always sync the dev-requirements.txt file (even if you only updated the requirements.in file) because the dev requirements inherit the requirements.txt too and you’re using the dev requirements locally (so the venv should always be updated to match any update)

  • Double-check that the venv has the correct updated version number
    pip freeze

  • Commit changes to Git
    git status
    git add -A
    git commit -m ‘upgraded OutdatedPackageName version to X.X.X; updated requirements files’

  • Double-check everythings OK by running the project’s tests again
    python -Wa manage.py test >>> use if using Django Test module
    PYTHONWARNINGS=always pytest --capture=no >>> use if using pytest

Upgrading Homebrew / Python checklist (Mac) [check every ~3 months]

  • Find the version of Python your project currently uses in venv
    python --version

  • Find the release date of the version currently used and compare it to the release date of the newer versions of Python
    Download Python | Python.org

  • Upgrade Python version (& other outdated stuff in Homebrew)

    • Ensure Homebrew is installed
      brew --version
    • Update Homebrew
      brew update
    • See outdated Homebrew packages
      brew outdated
    • Upgrade Homebrew packages
      brew upgrade
      Note: if desired, can do one at a time instead of all at once; more info

Some assumptions

  • Mac
  • pytest
  • pip-tools

Some sources

1 post - 1 participant

Read full topic

System check when both unique_together and UniqueConstraint are used
2 weeks, 5 days ago

Hi all,

With unique_together likely going away in the future in favour of UniqueConstraint, I noticed that Django does not raise a system check error or warning when you use both for the same set of fields on the same model. For example:

class TranslatableText(models.Model):
    translation_key = models.UUIDField(default=uuid.uuid4, editable=False)
    locale = models.CharField(max_length=8)
    content = models.TextField()

    class Meta:
        unique_together = [("translation_key", "locale")]
        constraints = [
                fields=["translation_key", "locale"],

It might be obvious that you shouldn’t do this. However, it’s not immediately obvious in the case of abstract models (e.g. from a third-party package that provides i18n) if you inherit the abstract model’s Meta as well. Without knowing that there’s a unique_together or a UniqueConstraint already in place, you might end up adding the other one and end up with two constraints for the same set of fields.

As a package maintainer, it’s pretty easy to add the system check yourself, but I’m wondering whether this should be included in Django itself. If so, I’d be happy to create a ticket and patch.

Although one might also argue it’s not worth to add the check into Django, as we’ll likely add another check to raise a deprecation warning about unique_together when it actually is going away anyway.

Another point is that Django also doesn’t raise an error if you have more than one UniqueConstraints with the same set of fields. This is understandable though, as UniqueConstraint has other options e.g. condition which means the constraints may have different purposes despite using the same fields. With unique_together, Django seems smart enough to consolidate duplicate entries into one when creating migrations (because they get converted to a set).

What do you think, shall we add such check(s)? Keen to hear your thoughts.


1 post - 1 participant

Read full topic

Debugging CSRF Failed / 403 Forbidden errors in Django
2 weeks, 6 days ago

I found a number of people, myself included run into csrf validation problems. And unfortunately they aren’t the easiest thing to debug. This is my attempt at helping someone walk through the error and dice into the Django codebase to understand and fix the issue.

Better Simple – 4 Nov 23

Debugging CSRF Failed / 403 Forbidden errors in Django

A guided deep dive into Django's source code to understand why your application is failing CSRF validation.

1 post - 1 participant

Read full topic

Database generated columns ⁽¹⁾: Django and SQLite
3 weeks ago

I published an introduction to database generated columns, using SQLite and the new GeneratedField added in Django 5.0

I’d be very happy to have your feedback on my article


1 post - 1 participant

Read full topic

aiGrunn: learntail, turn anything into a quiz using AI - Arjan Egges
2 weeks, 5 days ago

(One of my summaries of the 2023 Dutch aiGrunn AI conference in Groningen, NL).

Arjan is known for his programming videos.

Alternative title: "the dark side of integrating a LLM (large language model) in your software". You run into several challenges. He illustrates it with https://www.learntail.com/ , something he helped build. It creates quizes from text to make the reader more active.

What he used was the python library langchain to connect his app with a LLM. A handy trick: you can have it send extra format instructions to chatgpt based on a pydantic model. If it works, it works. But if you don't get proper json back, it crashes.

Some more challenges:

  • There is a limit on prompt length. If it gets too long, the LLM won't fully understand it anymore and ignore some of the instructions.
  • A LLM is no human being. So "hard" or "easy" don't mean anything. You have to be more machine-explicit, like "quiz without jargon".
  • The longest answer it provides is often the correct one. Because the data it has been trained on often has the longest one as the correct answer...
  • Limits are hard to predict. The token limit is input + output, so you basically have to know beforehand how many tokens the AI needs for its output.
  • Rate limiting is an issue. If you start chunking, for instance.

A LLM is not a proper API.

  • You need to do syntax checking on the answer.
  • Are all the fields present? Validation.
  • Are the answers of the right type (float/string/etc).

And hey, you can still write code yourself. You don't have to ask the LLM everything, you can just do the work yourself, too. An open question is whether developers will start to depend too much on LLMs.

aiGrunn: small and practical AI models for CO2 reduction in buildings - Bram de Wit
2 weeks, 5 days ago

(One of my summaries of the 2023 Dutch aiGrunn AI conference in Groningen, NL).

LLM models can be huge. Mind-boggling huge. But... we can also have fun with small models.

He works a company that regulates climate installations in buildings (HVAC, heating, ventilation, air conditioning) via the cloud. Buildings use 30% of all energy worldwide. So improving how the HVAC installation is used has a big impact.

A use case: normally you pre-heat rooms so that it is comfy when you arrive. But sometimes the sun quickly warms the room anyway shortly afterwards. Can you not conserve some energy without sacrificing too much comfort?

You could calculate an optimal solution, but "just" measuring every individual room in combination with an AI.

Technical setup:

  • An "edge device" inside the building.
  • An external API.
  • The API stores the data in mysql (the room metadata) and influxdb (the timeseries).
  • A user selects a room and a machine learning model type and a training data set (from historical data).
  • The software creates a dataset from influxdb, trains the model (pytorch). The trained neural network goes to ONNX (open neural network exchange). The output is stored in minio (S3-compatible object store). Note: all this is internal: no chatgpt or so.
  • With the business logic these predictions get interpreted and used for steering the heating. Normally you can achieve 3-5% savings.
  • The actual steering happens locally in the building with a "go" program that reads the ONNX data. It is open source and is called... gonnx :-)

They have a server with 1 GPU, which is enough for training all those models!

aiGrunn: fighting cancer with AI - Hylke Donker
2 weeks, 5 days ago

(One of my summaries of the 2023 Dutch aiGrunn AI conference in Groningen, NL).

What is cancer? According to wikipedia: abnormal cell growth with the potential to invade or spread to other parts of the body. That is what you can observe. Medically, there are several aspects of cancer:

  • It prevents the cell from dying.
  • It can grab more than usual resources.
  • No sensitivity to the regular anti-growth signals.
  • Etc.

AI starts getting used in clinics. For instance for proton therapy: where to best apply the proton radiation. And in radiology: letting AI look at images to detect cancer. A good AI can out-perform doctors. Analysis of blood samples, trying to detect cancer based on the DNA samples in there.

DNA mutations can also be detected, which is what he focuses on. Cancer is basically a "desease of the genome". DNA is made up of T, C, G and A sequences. Technically, it is perfectly feasable to "read" DNA.

How do mutations occur? Exposure can leave "scars" in DNA. Damage can occur due to sunlight or smoking for instance. Specific sources result in specific kinds of damage: smoking has a "preference" for changing specific letters. With analysis, you can thus detect/estimate the cause of cancer.

A method to detect it is non-negative matrix factorisation. Normally you can only summarize the data in "hard" clusters: something is either A or B. With this technique, you can do "soft" clusters: something can be a little bit A and a bit more B.

Matrix factorisation is a way to relate separate data sources. For movies, you can have persons with preferences for comedy or aciton movies. And movies with a percentage action/comedy. Combined you get a matrix with estimates for the preference for every movie per user.

In a similar way, he creates a matrix relating cancer causes (like smoking) to specific observed types of DNA damage.

But... how reliable are the results? You can treat the matrix as a neural network. You can then use bayesian analysis to assess the probabilities.

He made a python packge for his reasearch: "mubelnet" (though I couldn't find that online, btw).

AI is transforming cancer care. The only part it doesn't affect is the actual nursing process.

aiGrunn: thinking outside the chat box - JP van Oosten
2 weeks, 5 days ago

(One of my summaries of the 2023 Dutch aiGrunn AI conference in Groningen, NL).

Getting chatgpt to output valid json can be a chore:

> extract xxxx, output as json
> extract xxxx, output as json list
> extract xxxx, output as json with this schema
> extract xxxx, output as json, aargh JSON I BEG YOU

Apparently they solved the json problem last monday. But he had the same problem when trying to get chatgpt to output only English and not Dutch. So the underlying problem is still there: you have to beg it to output in a certain way and hope it listens.

Some other problems are hallucinations: chatgpt telling you something with complete confidence, even though being wrong. And biases. And it is not really a chatbot, as it doesn't ask questions. Unparseable output. Lack of explainability. Privacy issues as you're sending data to servers in the USA.

And... what are the data sources chatgpt used? We don't know. They're called "openAI", but they're definitively not open.

When to use LLMs and when not to use them. Some good use cases:

  • Zero/few shot learning. A quick way to get a simple minimum viable product or proof of concept.
  • Summarizing/transforming.
  • Data format transformation. html to json for instance.
  • You can use it to gather training data for easy bootstrapping.

Some bad use cases:

  • Structured classification tasks. You really want proper, neat output. Especially when you have lots of classes or a big context. For small personal projects it might be OK, but not for production.
  • Non-text classification... A large language model of course won't help you with it.
  • When costs or energy consumption is important. Scaling is an issue.
  • When it is unclear who is responsible for what gets outputted. A chatbot generating "of course, you can get a refund" can be problematic if the customer really wants the refund it should not get...
  • When you really want to be sure you get the right answer.

What are some ideas you can look at?

  • gzip plus near-neighbor analysis. Compress text and see how similar they are. It is not perfect, but it is a neat trick.
  • "Bag of words" plus "random forest" (a function from scipy).
  • Embeddings and a classifier. A LLM is used to annotate a dataset and you can then extract the interesting data and work with it.

What he thinks is important: keep humans in the loop. Prevent unwanted consequences. Add a preview step before sending stuff out into the world. Make classifications visible and allow corrections. Ask the user to label something if it is unclear. And don't forget to audit the automatic classifications.

When all you have is a LLM, everything might start to look like a generative task. But don't think like that. Who is going to use it? What is the actual problem? Spend some time thinking about it.

Django: Maybe disable PostgreSQL’s JIT to speed up many-joined queries
2 weeks, 6 days ago

Here’s a write-up of an optimization I made in my client Silvr’s project. I ended up disabling a PostgreSQL feature called the JIT (Just-In-Time) compiler which was taking a long time for little benefit.

This behaviour was observed on PostgreSQL 14, so things may have improved since. Although the version 15 and 16 release notes don’t seem to contain any relevant-looking changes.

Okay, story time…

I was looking at the project’s slowest tests and found one that took five seconds despite containing few lines of code. Profiling the test revealed a single slow query, which was also slow in production. I grabbed the query plan and fed it to pgMustard using the technique I previously blogged about):

pgMustard reported that JIT compilation of the query was taking 99.7% of the 3217ms runtime. The five-star rating for this finding meant that it was ripe for optimization. But why was JIT compilation taking so long?

The query used InheritanceManager from django-model-utils, which extends Django’s multi-table inheritance to return heterogenous subclasses of a base model:

>>> Place.objects.select_subclasses()
[<Library: Downtown Books>, <BookShop: Giovanni’s>]

The generated query joins all subclasses’ tables. With many subclasses, that query has many joins. But each join only adds a few rows to the results since each object is of only one leaf subclass.

However, PostgreSQL estimated that the large number of joins would mean many results, making it decide to JIT compile the query. JIT compiling a query takes a while, which is typically offset by the time savings. But with few results, this query took three seconds to JIT compile but milliseconds to execute.

pgMustard recommended either improving PostgreSQL’s configuration to reduce cost estimates to prevent JIT compiling, or disabling the JIT altogether. I found the number of cost estimation options bamboozling, so I gave up on that and tried disabling the JIT. I at least found a PostgreSQL mailing list post reporting similar JIT misbehaviour with low result counts, which was some reassurance that disabling the JIT wasn’t such a bad idea.

To disable the JIT, I extended the DATABASES setting’s OPTIONS with psycopg’s options to disable PostgreSQL’s jit setting, with a hefty explanatory query:

    "default": {
        "ENGINE": "django.db.backends.postgresql",
        # ...
        "OPTIONS": {
            # Disable PostgreSQL JIT
            # https://www.postgresql.org/docs/current/jit.html
            # django-model-utils’ select_subclasses() method joins a bunch of
            # tables, which PostgreSQL sees as high cost and JIT’s. But each
            # table will only match zero or one row per base class row, so this
            # cost estimate is vastly wrong, and JIT compiling costs a lot more
            # than it’s worth (e.g. 3.5 seconds with JIT vs 10 milliseconds
            # without). We’re not the only ones to encounter such a problem:
            # https://www.postgresql.org/message-id/70ec49f7-e9c4-bea7-c689-a4dfbdd66f78%40jurr.org
            # Using libq 'options' param, via psycopg:
            # https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-OPTIONS
            "options": "-c jit=off",

This option could also be set in the server configuration, but putting it in Django’s settings is fast and guaranteed to apply to all environments.

The change sped up the individual test to take milliseconds. Overall, the test suite went from 106 seconds to 94 seconds, an 11% improvement.

There was a risk of performance regressions in production, so we monitored when deploying. Thankfully, things only improved :) It seems the JIT wasn’t providing much value for this project.


Thanks once more to pgMustard—it truly cuts the mustard!


How to Kill the Django Development Server Without Ctrl+C
3 weeks, 2 days ago

Let's say you're developing a Django project and you've lost track of the terminal window that ran your python manage.py runserver command. If you can't press Ctrl+C in the terminal, how do you stop the server?

Finding the runserver Process

First, we need to find the ID of the manage.py runserver process.

On Linux, you can list all processes with the following command:

ps aux

To search the results, you can pipe the output of that command into the grep command:

ps aux | grep manage.py

If you want to search multiple words, enclose them in single quotes ' or double quotes ":

ps aux | grep 'manage.py runserver'

This should give you rows of table data such as:

lance    12839  0.0  0.1  57068 49840 pts/25   S+   07:11   0:00 python manage.py runserver
lance    12840 23.0  0.2 435156 67752 pts/25   Sl+  07:11   4:40 /home/lance/django-workout/.venv/bin/python manage.py runserver
lance    23752  0.0  0.0   4040  1980 pts/30   S+   07:31   0:00 grep --color=auto manage.py runserver

This can be confusing because there are a few numbers here and I'm not sure what each number represents.

To prepend a row of headers, we can run two commands back-to-back:

ps aux | head --lines=1 && ps aux | grep 'manage.py runserver'

The head command is used for printing the beginning of a file, or, if no file is specified, it reads standard input in the console.

We're looking for the PID or the "Process ID":

lance    12839  0.0  0.1  57068 49840 pts/25   S+   07:11   0:00 python manage.py runserver
lance    12840 23.0  0.2 435156 67752 pts/25   Sl+  07:11   4:40 /home/lance/django-workout/.venv/bin/python manage.py runserver
lance    23752  0.0  0.0   4040  1980 pts/30   S+   07:31   0:00 grep --color=auto manage.py runserver

There are three to pick from. Which do we choose?

We can kill either of the first two, PID 12839 or PID 12840. They are related. Notice that they have the same start time.

If we read the command from the third process, we can see it's a grep command. That's the one we just ran to search for results. That process is already gone by the time we're reading the table.

Killing a Process

Now that we know the process ID, we can tell the operating system to stop it:

kill 12839

Zero results suggests the command was successful, but we can double check.

Verify the Process was Terminated

We can confirm this worked by checking our process list again:

ps aux | head --lines=1 && ps aux | grep 'manage.py runserver'

You could also load up in your browser and see if your Django project is still being served there.


I had this happen to me when I started a server from the VSCode integrated terminal right when VSCode decided to restart.

Adding Full Text Search to Your Django App with django-watson
3 weeks, 3 days ago
Learn how to supercharge your Django app with full-text search using Django-Watson. Dive deep into Postgres magic and boost search functionality.
Django News - 204 No Content - Django 4.2.7, 4.1.13, and 3.2.23 Security Release - Nov 3rd 2023
3 weeks, 4 days ago


Django security releases issued: 4.2.7, 4.1.13, and 3.2.23

A new security release for a potential denial of service vulnerability in UsernameField on Windows. As ever, the best security practice is to keep your Django version up to date with the latest release.


Python Software Foundation News: Announcing our new Community Communications Manager!

The PSF's first Communications Manager will establish a PSF communications calendar, including annual messaging, newsletters, and blog posts. She will also partner with our Executive Director, Deb Nicholson, and other staffers to enhance our support for the Python community with various initiatives.


Updates to Django

Last week we had another massive 18 pull requests merged into Django by 13 different contributors - including 4 first time contributors! Congratulations to ksg, Leo Suarez, Izzy Hyman and CheesyPhoenix for having their first commit merged into Django - welcome on board!

The System check framework docs now has a new section on writing integration tests: System check framework - Writing integration tests. I always love to see the documentation continue to improve to better serve the community!

Coming in Django 5.1, there will be a new template tag {% query_string %} that allows changing a django.http.QueryDict instance for use in links, for example, to generate a link to the next page while keeping any filtering options in place. This should be hugely useful! You can read more about this here: query_string docs

Django Newsletter

Wagtail CMS

Wagtail 5.2 (LTS) release notes

Wagtail 5.2 is designated a Long Term Support (LTS) release which includes a redesigned page listing view, OpenSearch support, responsive & multi-format images with the picture tag, and much more.


Curate content better and make it greener in Wagtail 5.2

A nice overview of Wagtail 5.2's new features.


Sponsored Link

Get Expert Developer Support from the creators of Wagtail CMS

Get support for your developers from our team of Wagtail experts: from reviewing your Wagtail set-up and planning complex features, to code reviews, pair programming and more. We offer a range of plans, special pricing for charities and universities, and profits are invested in Wagtail product development.



New goodies in Django 5.0

From Django Fellow Mariusz Felisiak, an exploration of the "deluge" of amazing new features added in Django 5.0


Moving from Django Microframework to Django

A detailed look at recent efforts to create a single file Django "Hello, World" example and why analyzing those efforts demystifies some of the "magic" behind Django.


Customize the Django admin to differentiate environments

An elegant way to configure the admin colors based on production/staging/local environment.


Add infinite scroll with Django and HTMX in 60 seconds

Speed your initial load - get content only when the user wants it.


Toward sustainable development practices in open source

Mostly Python is a newsletter by Eric Matthes, the author of Python Crash Course and the django-simple-deploy package. This article talks about the new Djangonaut program and how it ties into sustainable open source.


Guide to setting up GeoDjango on Mac M1

How to setup GeoDjango using conda on an M-series Mac.


Not all open source work is equal

Python maintainer Hynek Schlaweck shares his thoughts on how being compensated for his work has helped him carve out more time to work on the open-source projects he maintains.



Updating our DEP process / DEP 1 - Django Internals

There is a good discussion on the Django Enhancement Proposal (DEP) process and how it might be updated and streamlined nearly a decade later.



DjangoCon Africa November 6-11

The first ever DjangoCon Africa is taking place in Zanzibar, Tanzania with six days of talks, workshops, education, and collaboration with the international Django community.


Python Pizza Hamburg (November 17th)

A microconference filled with pizza and 10-minute talks sharing lessons and experiences from real-world projects.


DjangoCon US 2023: Durham, NC Photos

DjangoCon US 2023 photos are live and were taken by Bartek Pawlik who has taken photos at many Django conferences and events all over the world going back a decade.


Sponsored Ad

Sick of performance issues?

Enter Scout's APM tool for Python apps. Easily pinpoint and fix slowdowns with intelligent tracing logic. Optimize performance hassle-free, delighting your users. Try us out for free!



Django Day CPH 2023: Paris Kasidiaris - Taming rich text content in Django

A talk on the challenges and opportunities of working with rich text in Django, and the development of the django-prose third-party package.


Django Day CPH 2023: Wilhelm Klopp on How to spend less time writing tests

This talk shares a novel approach to deterministically generating working integration tests from traces recorded using kolo. This reduces the time it takes to write tests overall and the often painful experience of setting up fixtures, test data, and database state.


VIDEO + Live Demo: Build ChatGPT-like Apps with AI

If you're interested in the practical applications of AI and Large Language Models (LLMs), you'll find value in this talk and live demo. The presentation goes beyond theory to include real-world examples and best practices, including a GitHub repository packed with Python code and ChatGPT-like app examples that will help you spin up your own app.


Python Community News: DjangoCon US 2023 Perspectives

A playlist with six and counting videos of DjangoCon US participants discussing the conference and their work in the Django community.


Django News Jobs

A few weeks ago, we launched Django News Jobs and there two new jobs this week.

Software Engineer - Ubuntu Systems Management at Canonical 🆕

Front End Web UI Django Developer (NC) at Ansys 🆕

Junior Web Developer at The Westervelt Company

Django Girls Communications Officer at Django Girls

Django Girls Awesomeness Ambassador at Django Girls

Senior Python/Django Engineer at Apexive

Senior Python/Django Software Engineer at Brivo Systems

Django Newsletter



Class-based emails for Django shipping with a test suite.


django-cms/django-sekizai: Django Template Blocks with extra functionality

Django Template Blocks with extra functionality.



What if we put HTMX into the Django Admin?


This RSS feed is published on https://django-news.com/. You can also subscribe via email.

Weeknotes (2023 week 44)
3 weeks, 5 days ago

Weeknotes (2023 week 44)

Unmaintained but maintained packages

There’s a discussion going on in the django-mptt issue tracker about the maintenance state of django-mptt. I have marked the project as unmaintained in March 2021 and haven’t regretted this decision at all. I haven’t had to fix inconsistencies in the tree structure once since switching to django-tree-queries. And if that wasn’t enough, I get little but only warm and thankful feedback for the latter, so that’s extra nice.

Despite marking django-mptt as unmaintained I seem to be doing a little bit of maintenance still. I’m still using it in old paid projects and so the things I do to make the package work for me is paid work. I’m not personally invested in the package anymore, so I’m able to tell people that there are absolutely no guarantees about the maintenance, and that feels good.

Read the Docs

I do understand why the .readthedocs.yaml file is now necessary. I wish that I wouldn’t have to do all the busywork of adding one to projects. I have just resubscribed to the Read the Docs Gold Membership which probably has expired at some point in the past. Read the Docs is excellent and everybody who can should support them.


  • feincms3 4.5, feincms3-sites 0.20 and feincms3-language-sites 0.3: Fixed the check which only allows adding an application through the CMS to the page tree (yes, that’s right) once; feincms3 worked fine, feincms3-language-sites by accident but feincms3-sites didn’t.
  • towel 0.31: Towel is one of my oldest packages which is still being used in real-world projects. Towel is a tool for building CRUD-type applications and is designed to keep you DRY while doing that. The project has been heavily inspired by a Django-based agency software I built many years back. The package even has docs! I’m still quite proud of the mostly transparent support for multitenancy, but apart from that I haven’t used it in many new projects.
2 weeks, 6 days ago

Langchain + Docker + Neo4j + Ollama

GenAI Stack

The GenAI Stack will get you started building your own GenAI application in no time. The demo applications can serve as inspiration or as a starting point. Learn more about the details in the technical blog post.


Create a .env file from the environment template file env.example

Available variables:

Variable Name Default value Description
OLLAMA_BASE_URL http://host.docker.internal:11434 REQUIRED - URL to Ollama LLM API
NEO4J_URI neo4j://database:7687 REQUIRED - URL to Neo4j database
NEO4J_USERNAME neo4j REQUIRED - Username for Neo4j database
NEO4J_PASSWORD password REQUIRED - Password for Neo4j database
LLM llama2 REQUIRED - Can be any Ollama model tag, or gpt-4 or gpt-3.5 or claudev2
EMBEDDING_MODEL sentence_transformer REQUIRED - Can be sentence_transformer, openai, aws or ollama
AWS_ACCESS_KEY_ID REQUIRED - Only if LLM=claudev2 or embedding_model=aws
AWS_SECRET_ACCESS_KEY REQUIRED - Only if LLM=claudev2 or embedding_model=aws
AWS_DEFAULT_REGION REQUIRED - Only if LLM=claudev2 or embedding_model=aws
OPENAI_API_KEY REQUIRED - Only if LLM=gpt-4 or LLM=gpt-3.5 or embedding_model=openai
LANGCHAIN_ENDPOINT "https://api.smith.langchain.com" OPTIONAL - URL to Langchain Smith API
LANGCHAIN_TRACING_V2 false OPTIONAL - Enable Langchain tracing v2
LANGCHAIN_PROJECT OPTIONAL - Langchain project name

LLM Configuration

MacOS and Linux users can use any LLM that's available via Ollama. Check the "tags" section under the model page you want to use on https://ollama.ai/library and write the tag for the value of the environment variable LLM= in th e.env file. All platforms can use GPT-3.5-turbo and GPT-4 (bring your own API keys for OpenAIs models).

MacOS Install Ollama on MacOS and start it before running docker compose up.

Linux No need to install Ollama manually, it will run in a container as part of the stack when running with the Linux profile: run docker compose --profile linux up. Make sure to set the OLLAMA_BASE_URL=http://llm:11434 in the .env file when using Ollama docker container.

To use the Linux-GPU profile: run docker compose --profile linux-gpu up. Also change OLLAMA_BASE_URL=http://llm-gpu:11434 in the .env file.

Windows Not supported by Ollama, so Windows users need to generate a OpenAI API key and configure the stack to use gpt-3.5 or gpt-4 in the .env file.


[!WARNING] There is a performance issue that impacts python applications in the 4.24.x releases of Docker Desktop. Please upgrade to the latest release before using this stack.

To start everything

docker compose up

If changes to build scripts has been made, rebuild.

docker compose up --build

To enter watch mode (auto rebuild on file changes). First start everything, then in new terminal:

docker compose watch

Shutdown If health check fails or containers don't start up as expected, shutdown completely to start up again.

docker compose down


Here's what's in this repo:

Name Main files Compose name URLs Description
Support Bot bot.py bot http://localhost:8501 Main usecase. Fullstack Python application.
Stack Overflow Loader loader.py loader http://localhost:8502 Load SO data into the database (create vector embeddings etc). Fullstack Python application.
PDF Reader pdf_bot.py pdf_bot http://localhost:8503 Read local PDF and ask it questions. Fullstack Python application.
Standalone Bot API api.py api http://localhost:8504 Standalone HTTP API streaming (SSE) + non-streaming endpoints. Python.
Standalone Bot UI front-end/ front-end http://localhost:8505 Standalone client that uses the Standalone Bot API to interact with the model. JavaScript (Svelte) front-end.

The database can be explored at http://localhost:7474.

App 1 - Support Agent Bot

UI: http://localhost:8501 DB client: http://localhost:7474

  • answer support question based on recent entries
  • provide summarized answers with sources
  • demonstrate difference between
    • RAG Disabled (pure LLM response)
    • RAG Enabled (vector + knowledge graph context)
  • allow to generate a high quality support ticket for the current conversation based on the style of highly rated questions in the database.

(Chat input + RAG mode selector)

(CTA to auto generate support ticket draft) (UI of the auto generated support ticket draft)

App 2 - Loader

UI: http://localhost:8502 DB client: http://localhost:7474

  • import recent Stack Overflow data for certain tags into a KG
  • embed questions and answers and store in vector index
  • UI: choose tags, run import, see progress, some stats of data in the database
  • Load high ranked questions (regardless of tags) to support the ticket generation feature of App 1.

App 3 Question / Answer with a local PDF

UI: http://localhost:8503
DB client: http://localhost:7474

This application lets you load a local PDF into text chunks and embed it into Neo4j so you can ask questions about its contents and have the LLM answer them using vector similarity search.

App 4 Standalone HTTP API


Example cURL command:

curl http://localhost:8504/query-stream\?text\=minimal%20hello%20world%20in%20python\&rag\=false

Exposes the functionality to answer questions in the same way as App 1 above. Uses same code and prompts.

App 5 Static front-end

UI: http://localhost:8505

This application has the same features as App 1, but is built separate from the back-end code using modern best practices (Vite, Svelte, Tailwind).
The auto-reload on changes are instant using the Docker watch sync config.

3 weeks, 2 days ago

PyTorch image models, scripts, pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNet-V3/V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

PyTorch Image Models

What's New

❗Updates after Oct 10, 2022 are available in version >= 0.9❗

  • Many changes since the last 0.6.x stable releases. They were previewed in 0.8.x dev releases but not everyone transitioned.
  • timm.models.layers moved to timm.layers:
    • from timm.models.layers import name will still work via deprecation mapping (but please transition to timm.layers).
    • import timm.models.layers.module or from timm.models.layers.module import name needs to be changed now.
  • Builder, helper, non-model modules in timm.models have a _ prefix added, ie timm.models.helpers -> timm.models._helpers, there are temporary deprecation mapping files but those will be removed.
  • All models now support architecture.pretrained_tag naming (ex resnet50.rsb_a1).
    • The pretrained_tag is the specific weight variant (different head) for the architecture.
    • Using only architecture defaults to the first weights in the default_cfgs for that model architecture.
    • In adding pretrained tags, many model names that existed to differentiate were renamed to use the tag (ex: vit_base_patch16_224_in21k -> vit_base_patch16_224.augreg_in21k). There are deprecation mappings for these.
  • A number of models had their checkpoints remaped to match architecture changes needed to better support features_only=True, there are checkpoint_filter_fn methods in any model module that was remapped. These can be passed to timm.models.load_checkpoint(..., filter_fn=timm.models.swin_transformer_v2.checkpoint_filter_fn) to remap your existing checkpoint.
  • The Hugging Face Hub (https://huggingface.co/timm) is now the primary source for timm weights. Model cards include link to papers, original source, license.
  • Previous 0.6.x can be cloned from 0.6.x branch or installed via pip with version.

Nov 3, 2023

  • DFN (Data Filtering Networks) and MetaCLIP ViT weights added
  • DINOv2 'register' ViT model weights added
  • Add quickgelu ViT variants for OpenAI, DFN, MetaCLIP weights that use it (less efficient)
  • Improved typing added to ResNet, MobileNet-v3 thanks to Aryan
  • ImageNet-12k fine-tuned (from LAION-2B CLIP) convnext_xxlarge
  • 0.9.9 release

Oct 20, 2023

  • SigLIP image tower weights supported in vision_transformer.py.
    • Great potential for fine-tune and downstream feature use.
  • Experimental 'register' support in vit models as per Vision Transformers Need Registers
  • Updated RepViT with new weight release. Thanks wangao
  • Add patch resizing support (on pretrained weight load) to Swin models
  • 0.9.8 release pending

Sep 1, 2023

  • TinyViT added by SeeFun
  • Fix EfficientViT (MIT) to use torch.autocast so it works back to PT 1.10
  • 0.9.7 release

Aug 28, 2023

  • Add dynamic img size support to models in vision_transformer.py, vision_transformer_hybrid.py, deit.py, and eva.py w/o breaking backward compat.
    • Add dynamic_img_size=True to args at model creation time to allow changing the grid size (interpolate abs and/or ROPE pos embed each forward pass).
    • Add dynamic_img_pad=True to allow image sizes that aren't divisible by patch size (pad bottom right to patch size each forward pass).
    • Enabling either dynamic mode will break FX tracing unless PatchEmbed module added as leaf.
    • Existing method of resizing position embedding by passing different img_size (interpolate pretrained embed weights once) on creation still works.
    • Existing method of changing patch_size (resize pretrained patch_embed weights once) on creation still works.
    • Example validation cmd python validate.py /imagenet --model vit_base_patch16_224 --amp --amp-dtype bfloat16 --img-size 255 --crop-pct 1.0 --model-kwargs dynamic_img_size=True dyamic_img_pad=True

Aug 25, 2023

Aug 11, 2023

  • Swin, MaxViT, CoAtNet, and BEiT models support resizing of image/window size on creation with adaptation of pretrained weights
  • Example validation cmd to test w/ non-square resize python validate.py /imagenet --model swin_base_patch4_window7_224.ms_in22k_ft_in1k --amp --amp-dtype bfloat16 --input-size 3 256 320 --model-kwargs window_size=8,10 img_size=256,320

Aug 3, 2023

  • Add GluonCV weights for HRNet w18_small and w18_small_v2. Converted by SeeFun
  • Fix selecsls* model naming regression
  • Patch and position embedding for ViT/EVA works for bfloat16/float16 weights on load (or activations for on-the-fly resize)
  • v0.9.5 release prep

July 27, 2023

  • Added timm trained seresnextaa201d_32x8d.sw_in12k_ft_in1k_384 weights (and .sw_in12k pretrain) with 87.3% top-1 on ImageNet-1k, best ImageNet ResNet family model I'm aware of.
  • RepViT model and weights (https://arxiv.org/abs/2307.09283) added by wangao
  • I-JEPA ViT feature weights (no classifier) added by SeeFun
  • SAM-ViT (segment anything) feature weights (no classifier) added by SeeFun
  • Add support for alternative feat extraction methods and -ve indices to EfficientNet
  • Add NAdamW optimizer
  • Misc fixes

May 11, 2023

  • timm 0.9 released, transition from 0.8.xdev releases

May 10, 2023

  • Hugging Face Hub downloading is now default, 1132 models on https://huggingface.co/timm, 1163 weights in timm
  • DINOv2 vit feature backbone weights added thanks to Leng Yue
  • FB MAE vit feature backbone weights added
  • OpenCLIP DataComp-XL L/14 feat backbone weights added
  • MetaFormer (poolformer-v2, caformer, convformer, updated poolformer (v1)) w/ weights added by Fredo Guan
  • Experimental get_intermediate_layers function on vit/deit models for grabbing hidden states (inspired by DINO impl). This is WIP and may change significantly... feedback welcome.
  • Model creation throws error if pretrained=True and no weights exist (instead of continuing with random initialization)
  • Fix regression with inception / nasnet TF sourced weights with 1001 classes in original classifiers
  • bitsandbytes (https://github.com/TimDettmers/bitsandbytes) optimizers added to factory, use bnb prefix, ie bnbadam8bit
  • Misc cleanup and fixes
  • Final testing before switching to a 0.9 and bringing timm out of pre-release state

April 27, 2023

  • 97% of timm models uploaded to HF Hub and almost all updated to support multi-weight pretrained configs
  • Minor cleanup and refactoring of another batch of models as multi-weight added. More fused_attn (F.sdpa) and features_only support, and torchscript fixes.

April 21, 2023

  • Gradient accumulation support added to train script and tested (--grad-accum-steps), thanks Taeksang Kim
  • More weights on HF Hub (cspnet, cait, volo, xcit, tresnet, hardcorenas, densenet, dpn, vovnet, xception_aligned)
  • Added --head-init-scale and --head-init-bias to train.py to scale classiifer head and set fixed bias for fine-tune
  • Remove all InplaceABN (inplace_abn) use, replaced use in tresnet with standard BatchNorm (modified weights accordingly).

April 12, 2023

  • Add ONNX export script, validate script, helpers that I've had kicking around for along time. Tweak 'same' padding for better export w/ recent ONNX + pytorch.
  • Refactor dropout args for vit and vit-like models, separate drop_rate into drop_rate (classifier dropout), proj_drop_rate (block mlp / out projections), pos_drop_rate (position embedding drop), attn_drop_rate (attention dropout). Also add patch dropout (FLIP) to vit and eva models.
  • fused F.scaled_dot_product_attention support to more vit models, add env var (TIMM_FUSED_ATTN) to control, and config interface to enable/disable
  • Add EVA-CLIP backbones w/ image tower weights, all the way up to 4B param 'enormous' model, and 336x336 OpenAI ViT mode that was missed.

April 5, 2023

  • ALL ResNet models pushed to Hugging Face Hub with multi-weight support
  • New ImageNet-12k + ImageNet-1k fine-tunes available for a few anti-aliased ResNet models
    • resnetaa50d.sw_in12k_ft_in1k - 81.7 @ 224, 82.6 @ 288
    • resnetaa101d.sw_in12k_ft_in1k - 83.5 @ 224, 84.1 @ 288
    • seresnextaa101d_32x8d.sw_in12k_ft_in1k - 86.0 @ 224, 86.5 @ 288
    • seresnextaa101d_32x8d.sw_in12k_ft_in1k_288 - 86.5 @ 288, 86.7 @ 320

March 31, 2023

  • Add first ConvNext-XXLarge CLIP -> IN-1k fine-tune and IN-12k intermediate fine-tunes for convnext-base/large CLIP models.
model top1 top5 img_size param_count gmacs macts
convnext_xxlarge.clip_laion2b_soup_ft_in1k 88.612 98.704 256 846.47 198.09 124.45
convnext_large_mlp.clip_laion2b_soup_ft_in12k_in1k_384 88.312 98.578 384 200.13 101.11 126.74
convnext_large_mlp.clip_laion2b_soup_ft_in12k_in1k_320 87.968 98.47 320 200.13 70.21 88.02
convnext_base.clip_laion2b_augreg_ft_in12k_in1k_384 87.138 98.212 384 88.59 45.21 84.49
convnext_base.clip_laion2b_augreg_ft_in12k_in1k 86.344 97.97 256 88.59 20.09 37.55
  • Add EVA-02 MIM pretrained and fine-tuned weights, push to HF hub and update model cards for all EVA models. First model over 90% top-1 (99% top-5)! Check out the original code & weights at https://github.com/baaivision/EVA for more details on their work blending MIM, CLIP w/ many model, dataset, and train recipe tweaks.
model top1 top5 param_count img_size
eva02_large_patch14_448.mim_m38m_ft_in22k_in1k 90.054 99.042 305.08 448
eva02_large_patch14_448.mim_in22k_ft_in22k_in1k 89.946 99.01 305.08 448
eva_giant_patch14_560.m30m_ft_in22k_in1k 89.792 98.992 1014.45 560
eva02_large_patch14_448.mim_in22k_ft_in1k 89.626 98.954 305.08 448
eva02_large_patch14_448.mim_m38m_ft_in1k 89.57 98.918 305.08 448
eva_giant_patch14_336.m30m_ft_in22k_in1k 89.56 98.956 1013.01 336
eva_giant_patch14_336.clip_ft_in1k 89.466 98.82 1013.01 336
eva_large_patch14_336.in22k_ft_in22k_in1k 89.214 98.854 304.53 336
eva_giant_patch14_224.clip_ft_in1k 88.882 98.678 1012.56 224
eva02_base_patch14_448.mim_in22k_ft_in22k_in1k 88.692 98.722 87.12 448
eva_large_patch14_336.in22k_ft_in1k 88.652 98.722 304.53 336
eva_large_patch14_196.in22k_ft_in22k_in1k 88.592 98.656 304.14 196
eva02_base_patch14_448.mim_in22k_ft_in1k 88.23 98.564 87.12 448
eva_large_patch14_196.in22k_ft_in1k 87.934 98.504 304.14 196
eva02_small_patch14_336.mim_in22k_ft_in1k 85.74 97.614 22.13 336
eva02_tiny_patch14_336.mim_in22k_ft_in1k 80.658 95.524 5.76 336
  • Multi-weight and HF hub for DeiT and MLP-Mixer based models

March 22, 2023

  • More weights pushed to HF hub along with multi-weight support, including: regnet.py, rexnet.py, byobnet.py, resnetv2.py, swin_transformer.py, swin_transformer_v2.py, swin_transformer_v2_cr.py
  • Swin Transformer models support feature extraction (NCHW feat maps for swinv2_cr_*, and NHWC for all others) and spatial embedding outputs.
  • FocalNet (from https://github.com/microsoft/FocalNet) models and weights added with significant refactoring, feature extraction, no fixed resolution / sizing constraint
  • RegNet weights increased with HF hub push, SWAG, SEER, and torchvision v2 weights. SEER is pretty poor wrt to performance for model size, but possibly useful.
  • More ImageNet-12k pretrained and 1k fine-tuned timm weights:
    • rexnetr_200.sw_in12k_ft_in1k - 82.6 @ 224, 83.2 @ 288
    • rexnetr_300.sw_in12k_ft_in1k - 84.0 @ 224, 84.5 @ 288
    • regnety_120.sw_in12k_ft_in1k - 85.0 @ 224, 85.4 @ 288
    • regnety_160.lion_in12k_ft_in1k - 85.6 @ 224, 86.0 @ 288
    • regnety_160.sw_in12k_ft_in1k - 85.6 @ 224, 86.0 @ 288 (compare to SWAG PT + 1k FT this is same BUT much lower res, blows SEER FT away)
  • Model name deprecation + remapping functionality added (a milestone for bringing 0.8.x out of pre-release). Mappings being added...
  • Minor bug fixes and improvements.

Feb 26, 2023

  • Add ConvNeXt-XXLarge CLIP pretrained image tower weights for fine-tune & features (fine-tuning TBD) -- see model card
  • Update convnext_xxlarge default LayerNorm eps to 1e-5 (for CLIP weights, improved stability)
  • 0.8.15dev0

Feb 20, 2023

  • Add 320x320 convnext_large_mlp.clip_laion2b_ft_320 and convnext_lage_mlp.clip_laion2b_ft_soup_320 CLIP image tower weights for features & fine-tune
  • 0.8.13dev0 pypi release for latest changes w/ move to huggingface org

Feb 16, 2023

  • safetensor checkpoint support added
  • Add ideas from 'Scaling Vision Transformers to 22 B. Params' (https://arxiv.org/abs/2302.05442) -- qk norm, RmsNorm, parallel block
  • Add F.scaled_dot_product_attention support (PyTorch 2.0 only) to vit_*, vit_relpos*, coatnet / maxxvit (to start)
  • Lion optimizer (w/ multi-tensor option) added (https://arxiv.org/abs/2302.06675)
  • gradient checkpointing works with features_only=True

Feb 7, 2023

  • New inference benchmark numbers added in results folder.
  • Add convnext LAION CLIP trained weights and initial set of in1k fine-tunes
    • convnext_base.clip_laion2b_augreg_ft_in1k - 86.2% @ 256x256
    • convnext_base.clip_laiona_augreg_ft_in1k_384 - 86.5% @ 384x384
    • convnext_large_mlp.clip_laion2b_augreg_ft_in1k - 87.3% @ 256x256
    • convnext_large_mlp.clip_laion2b_augreg_ft_in1k_384 - 87.9% @ 384x384
  • Add DaViT models. Supports features_only=True. Adapted from https://github.com/dingmyu/davit by Fredo.
  • Use a common NormMlpClassifierHead across MaxViT, ConvNeXt, DaViT
  • Add EfficientFormer-V2 model, update EfficientFormer, and refactor LeViT (closely related architectures). Weights on HF hub.
    • New EfficientFormer-V2 arch, significant refactor from original at (https://github.com/snap-research/EfficientFormer). Supports features_only=True.
    • Minor updates to EfficientFormer.
    • Refactor LeViT models to stages, add features_only=True support to new conv variants, weight remap required.
  • Move ImageNet meta-data (synsets, indices) from /results to timm/data/_info.
  • Add ImageNetInfo / DatasetInfo classes to provide labelling for various ImageNet classifier layouts in timm
    • Update inference.py to use, try: python inference.py /folder/to/images --model convnext_small.in12k --label-type detail --topk 5
  • Ready for 0.8.10 pypi pre-release (final testing).

Jan 20, 2023

  • Add two convnext 12k -> 1k fine-tunes at 384x384

    • convnext_tiny.in12k_ft_in1k_384 - 85.1 @ 384
    • convnext_small.in12k_ft_in1k_384 - 86.2 @ 384
  • Push all MaxxViT weights to HF hub, and add new ImageNet-12k -> 1k fine-tunes for rw base MaxViT and CoAtNet 1/2 models

model top1 top5 samples / sec Params (M) GMAC Act (M)
maxvit_xlarge_tf_512.in21k_ft_in1k 88.53 98.64 21.76 475.77 534.14 1413.22
maxvit_xlarge_tf_384.in21k_ft_in1k 88.32 98.54 42.53 475.32 292.78 668.76
maxvit_base_tf_512.in21k_ft_in1k 88.20 98.53 50.87 119.88 138.02 703.99
maxvit_large_tf_512.in21k_ft_in1k 88.04 98.40 36.42 212.33 244.75 942.15
maxvit_large_tf_384.in21k_ft_in1k 87.98 98.56 71.75 212.03 132.55 445.84
maxvit_base_tf_384.in21k_ft_in1k 87.92 98.54 104.71 119.65 73.80 332.90
maxvit_rmlp_base_rw_384.sw_in12k_ft_in1k 87.81 98.37 106.55 116.14 70.97 318.95
maxxvitv2_rmlp_base_rw_384.sw_in12k_ft_in1k 87.47 98.37 149.49 116.09 72.98 213.74
coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k 87.39 98.31 160.80 73.88 47.69 209.43
maxvit_rmlp_base_rw_224.sw_in12k_ft_in1k 86.89 98.02 375.86 116.14 23.15 92.64
maxxvitv2_rmlp_base_rw_224.sw_in12k_ft_in1k 86.64 98.02 501.03 116.09 24.20 62.77
maxvit_base_tf_512.in1k 86.60 97.92 50.75 119.88 138.02 703.99
coatnet_2_rw_224.sw_in12k_ft_in1k 86.57 97.89 631.88 73.87 15.09 49.22
maxvit_large_tf_512.in1k 86.52 97.88 36.04 212.33 244.75 942.15
coatnet_rmlp_2_rw_224.sw_in12k_ft_in1k 86.49 97.90 620.58 73.88 15.18 54.78
maxvit_base_tf_384.in1k 86.29 97.80 101.09 119.65 73.80 332.90
maxvit_large_tf_384.in1k 86.23 97.69 70.56 212.03 132.55 445.84
maxvit_small_tf_512.in1k 86.10 97.76 88.63 69.13 67.26 383.77
maxvit_tiny_tf_512.in1k 85.67 97.58 144.25 31.05 33.49 257.59
maxvit_small_tf_384.in1k 85.54 97.46 188.35 69.02 35.87 183.65
maxvit_tiny_tf_384.in1k 85.11 97.38 293.46 30.98 17.53 123.42
maxvit_large_tf_224.in1k 84.93 96.97 247.71 211.79 43.68 127.35
coatnet_rmlp_1_rw2_224.sw_in12k_ft_in1k 84.90 96.96 1025.45 41.72 8.11 40.13
maxvit_base_tf_224.in1k 84.85 96.99 358.25 119.47 24.04 95.01
maxxvit_rmlp_small_rw_256.sw_in1k 84.63 97.06 575.53 66.01 14.67 58.38
coatnet_rmlp_2_rw_224.sw_in1k 84.61 96.74 625.81 73.88 15.18 54.78
maxvit_rmlp_small_rw_224.sw_in1k 84.49 96.76 693.82 64.90 10.75 49.30
maxvit_small_tf_224.in1k 84.43 96.83 647.96 68.93 11.66 53.17
maxvit_rmlp_tiny_rw_256.sw_in1k 84.23 96.78 807.21 29.15 6.77 46.92
coatnet_1_rw_224.sw_in1k 83.62 96.38 989.59 41.72 8.04 34.60
maxvit_tiny_rw_224.sw_in1k 83.50 96.50 1100.53 29.06 5.11 33.11
maxvit_tiny_tf_224.in1k 83.41 96.59 1004.94 30.92 5.60 35.78
coatnet_rmlp_1_rw_224.sw_in1k 83.36 96.45 1093.03 41.69 7.85 35.47
maxxvitv2_nano_rw_256.sw_in1k 83.11 96.33 1276.88 23.70 6.26 23.05
maxxvit_rmlp_nano_rw_256.sw_in1k 83.03 96.34 1341.24 16.78 4.37 26.05
maxvit_rmlp_nano_rw_256.sw_in1k 82.96 96.26 1283.24 15.50 4.47 31.92
maxvit_nano_rw_256.sw_in1k 82.93 96.23 1218.17 15.45 4.46 30.28
coatnet_bn_0_rw_224.sw_in1k 82.39 96.19 1600.14 27.44 4.67 22.04
coatnet_0_rw_224.sw_in1k 82.39 95.84 1831.21 27.44 4.43 18.73
coatnet_rmlp_nano_rw_224.sw_in1k 82.05 95.87 2109.09 15.15 2.62 20.34
coatnext_nano_rw_224.sw_in1k 81.95 95.92 2525.52 14.70 2.47 12.80
coatnet_nano_rw_224.sw_in1k 81.70 95.64 2344.52 15.14 2.41 15.41
maxvit_rmlp_pico_rw_256.sw_in1k 80.53 95.21 1594.71 7.52 1.85 24.86

Jan 11, 2023

  • Update ConvNeXt ImageNet-12k pretrain series w/ two new fine-tuned weights (and pre FT .in12k tags)
    • convnext_nano.in12k_ft_in1k - 82.3 @ 224, 82.9 @ 288 (previously released)
    • convnext_tiny.in12k_ft_in1k - 84.2 @ 224, 84.5 @ 288
    • convnext_small.in12k_ft_in1k - 85.2 @ 224, 85.3 @ 288

Jan 6, 2023

  • Finally got around to adding --model-kwargs and --opt-kwargs to scripts to pass through rare args directly to model classes from cmd line
    • train.py /imagenet --model resnet50 --amp --model-kwargs output_stride=16 act_layer=silu
    • train.py /imagenet --model vit_base_patch16_clip_224 --img-size 240 --amp --model-kwargs img_size=240 patch_size=12
  • Cleanup some popular models to better support arg passthrough / merge with model configs, more to go.

Jan 5, 2023

Dec 23, 2022 🎄☃

  • Add FlexiViT models and weights from https://github.com/google-research/big_vision (check out paper at https://arxiv.org/abs/2212.08013)
    • NOTE currently resizing is static on model creation, on-the-fly dynamic / train patch size sampling is a WIP
  • Many more models updated to multi-weight and downloadable via HF hub now (convnext, efficientnet, mobilenet, vision_transformer*, beit)
  • More model pretrained tag and adjustments, some model names changed (working on deprecation translations, consider main branch DEV branch right now, use 0.6.x for stable use)
  • More ImageNet-12k (subset of 22k) pretrain models popping up:
    • efficientnet_b5.in12k_ft_in1k - 85.9 @ 448x448
    • vit_medium_patch16_gap_384.in12k_ft_in1k - 85.5 @ 384x384
    • vit_medium_patch16_gap_256.in12k_ft_in1k - 84.5 @ 256x256
    • convnext_nano.in12k_ft_in1k - 82.9 @ 288x288

Dec 8, 2022

  • Add 'EVA l' to vision_transformer.py, MAE style ViT-L/14 MIM pretrain w/ EVA-CLIP targets, FT on ImageNet-1k (w/ ImageNet-22k intermediate for some)
model top1 param_count gmac macts hub
eva_large_patch14_336.in22k_ft_in22k_in1k 89.2 304.5 191.1 270.2 link
eva_large_patch14_336.in22k_ft_in1k 88.7 304.5 191.1 270.2 link
eva_large_patch14_196.in22k_ft_in22k_in1k 88.6 304.1 61.6 63.5 link
eva_large_patch14_196.in22k_ft_in1k 87.9 304.1 61.6 63.5 link

Dec 6, 2022

model top1 param_count gmac macts hub
eva_giant_patch14_560.m30m_ft_in22k_in1k 89.8 1014.4 1906.8 2577.2 link
eva_giant_patch14_336.m30m_ft_in22k_in1k 89.6 1013 620.6 550.7 link
eva_giant_patch14_336.clip_ft_in1k 89.4 1013 620.6 550.7 link
eva_giant_patch14_224.clip_ft_in1k 89.1 1012.6 267.2 192.6 link

Dec 5, 2022

  • Pre-release (0.8.0dev0) of multi-weight support (model_arch.pretrained_tag). Install with pip install --pre timm
    • vision_transformer, maxvit, convnext are the first three model impl w/ support
    • model names are changing with this (previous _21k, etc. fn will merge), still sorting out deprecation handling
    • bugs are likely, but I need feedback so please try it out
    • if stability is needed, please use 0.6.x pypi releases or clone from 0.6.x branch
  • Support for PyTorch 2.0 compile is added in train/validate/inference/benchmark, use --torchcompile argument
  • Inference script allows more control over output, select k for top-class index + prob json, csv or parquet output
  • Add a full set of fine-tuned CLIP image tower weights from both LAION-2B and original OpenAI CLIP models
model top1 param_count gmac macts hub
vit_huge_patch14_clip_336.laion2b_ft_in12k_in1k 88.6 632.5 391 407.5 link
vit_large_patch14_clip_336.openai_ft_in12k_in1k 88.3 304.5 191.1 270.2 link
vit_huge_patch14_clip_224.laion2b_ft_in12k_in1k 88.2 632 167.4 139.4 link
vit_large_patch14_clip_336.laion2b_ft_in12k_in1k 88.2 304.5 191.1 270.2 link
vit_large_patch14_clip_224.openai_ft_in12k_in1k 88.2 304.2 81.1 88.8 link
vit_large_patch14_clip_224.laion2b_ft_in12k_in1k 87.9 304.2 81.1 88.8 link
vit_large_patch14_clip_224.openai_ft_in1k 87.9 304.2 81.1 88.8 link
vit_large_patch14_clip_336.laion2b_ft_in1k 87.9 304.5 191.1 270.2 link
vit_huge_patch14_clip_224.laion2b_ft_in1k 87.6 632 167.4 139.4 link
vit_large_patch14_clip_224.laion2b_ft_in1k 87.3 304.2 81.1 88.8 link
vit_base_patch16_clip_384.laion2b_ft_in12k_in1k 87.2 86.9 55.5 101.6 link
vit_base_patch16_clip_384.openai_ft_in12k_in1k 87 86.9 55.5 101.6 link
vit_base_patch16_clip_384.laion2b_ft_in1k 86.6 86.9 55.5 101.6 link
vit_base_patch16_clip_384.openai_ft_in1k 86.2 86.9 55.5 101.6 link
vit_base_patch16_clip_224.laion2b_ft_in12k_in1k 86.2 86.6 17.6 23.9 link
vit_base_patch16_clip_224.openai_ft_in12k_in1k 85.9 86.6 17.6 23.9 link
vit_base_patch32_clip_448.laion2b_ft_in12k_in1k 85.8 88.3 17.9 23.9 link
vit_base_patch16_clip_224.laion2b_ft_in1k 85.5 86.6 17.6 23.9 link
vit_base_patch32_clip_384.laion2b_ft_in12k_in1k 85.4 88.3 13.1 16.5 link
vit_base_patch16_clip_224.openai_ft_in1k 85.3 86.6 17.6 23.9 link
vit_base_patch32_clip_384.openai_ft_in12k_in1k 85.2 88.3 13.1 16.5 link
vit_base_patch32_clip_224.laion2b_ft_in12k_in1k 83.3 88.2 4.4 5 link
vit_base_patch32_clip_224.laion2b_ft_in1k 82.6 88.2 4.4 5 link
vit_base_patch32_clip_224.openai_ft_in1k 81.9 88.2 4.4 5 link
  • Port of MaxViT Tensorflow Weights from official impl at https://github.com/google-research/maxvit
    • There was larger than expected drops for the upscaled 384/512 in21k fine-tune weights, possible detail missing, but the 21k FT did seem sensitive to small preprocessing
model top1 param_count gmac macts hub
maxvit_xlarge_tf_512.in21k_ft_in1k 88.5 475.8 534.1 1413.2 link
maxvit_xlarge_tf_384.in21k_ft_in1k 88.3 475.3 292.8 668.8 link
maxvit_base_tf_512.in21k_ft_in1k 88.2 119.9 138 704 link
maxvit_large_tf_512.in21k_ft_in1k 88 212.3 244.8 942.2 link
maxvit_large_tf_384.in21k_ft_in1k 88 212 132.6 445.8 link
maxvit_base_tf_384.in21k_ft_in1k 87.9 119.6 73.8 332.9 link
maxvit_base_tf_512.in1k 86.6 119.9 138 704 link
maxvit_large_tf_512.in1k 86.5 212.3 244.8 942.2 link
maxvit_base_tf_384.in1k 86.3 119.6 73.8 332.9 link
maxvit_large_tf_384.in1k 86.2 212 132.6 445.8 link
maxvit_small_tf_512.in1k 86.1 69.1 67.3 383.8 link
maxvit_tiny_tf_512.in1k 85.7 31 33.5 257.6 link
maxvit_small_tf_384.in1k 85.5 69 35.9 183.6 link
maxvit_tiny_tf_384.in1k 85.1 31 17.5 123.4 link
maxvit_large_tf_224.in1k 84.9 211.8 43.7 127.4 link
maxvit_base_tf_224.in1k 84.9 119.5 24 95 link
maxvit_small_tf_224.in1k 84.4 68.9 11.7 53.2 link
maxvit_tiny_tf_224.in1k 83.4 30.9 5.6 35.8 link

Oct 15, 2022

  • Train and validation script enhancements
  • Non-GPU (ie CPU) device support
  • SLURM compatibility for train script
  • HF datasets support (via ReaderHfds)
  • TFDS/WDS dataloading improvements (sample padding/wrap for distributed use fixed wrt sample count estimate)
  • in_chans !=3 support for scripts / loader
  • Adan optimizer
  • Can enable per-step LR scheduling via args
  • Dataset 'parsers' renamed to 'readers', more descriptive of purpose
  • AMP args changed, APEX via --amp-impl apex, bfloat16 supportedf via --amp-dtype bfloat16
  • main branch switched to 0.7.x version, 0.6x forked for stable release of weight only adds
  • master -> main branch rename

Oct 10, 2022

  • More weights in maxxvit series, incl first ConvNeXt block based coatnext and maxxvit experiments:
    • coatnext_nano_rw_224 - 82.0 @ 224 (G) -- (uses ConvNeXt conv block, no BatchNorm)
    • maxxvit_rmlp_nano_rw_256 - 83.0 @ 256, 83.7 @ 320 (G) (uses ConvNeXt conv block, no BN)
    • maxvit_rmlp_small_rw_224 - 84.5 @ 224, 85.1 @ 320 (G)
    • maxxvit_rmlp_small_rw_256 - 84.6 @ 256, 84.9 @ 288 (G) -- could be trained better, hparams need tuning (uses ConvNeXt block, no BN)
    • coatnet_rmlp_2_rw_224 - 84.6 @ 224, 85 @ 320 (T)
    • NOTE: official MaxVit weights (in1k) have been released at https://github.com/google-research/maxvit -- some extra work is needed to port and adapt since my impl was created independently of theirs and has a few small differences + the whole TF same padding fun.

Sept 23, 2022

  • LAION-2B CLIP image towers supported as pretrained backbones for fine-tune or features (no classifier)
    • vit_base_patch32_224_clip_laion2b
    • vit_large_patch14_224_clip_laion2b
    • vit_huge_patch14_224_clip_laion2b
    • vit_giant_patch14_224_clip_laion2b

Sept 7, 2022

  • Hugging Face timm docs home now exists, look for more here in the future
  • Add BEiT-v2 weights for base and large 224x224 models from https://github.com/microsoft/unilm/tree/master/beit2
  • Add more weights in maxxvit series incl a pico (7.5M params, 1.9 GMACs), two tiny variants:
    • maxvit_rmlp_pico_rw_256 - 80.5 @ 256, 81.3 @ 320 (T)
    • maxvit_tiny_rw_224 - 83.5 @ 224 (G)
    • maxvit_rmlp_tiny_rw_256 - 84.2 @ 256, 84.8 @ 320 (T)


PyTorch Image Models (timm) is a collection of image models, layers, utilities, optimizers, schedulers, data-loaders / augmentations, and reference training / validation scripts that aim to pull together a wide variety of SOTA models with ability to reproduce ImageNet training results.

The work of many others is present here. I've tried to make sure all source material is acknowledged via links to github, arxiv papers, etc in the README, documentation, and code docstrings. Please let me know if I missed anything.


All model architecture families include variants with pretrained weights. There are specific model variants without any weights, it is NOT a bug. Help training new or better weights is always appreciated.


Several (less common) features that I often utilize in my projects are included. Many of their additions are the reason why I maintain my own set of models, instead of using others' via PIP:


Model validation results can be found in the results tables

Getting Started (Documentation)

The official documentation can be found at https://huggingface.co/docs/hub/timm. Documentation contributions are welcome.

Getting Started with PyTorch Image Models (timm): A Practitioner’s Guide by Chris Hughes is an extensive blog post covering many aspects of timm in detail.

timmdocs is an alternate set of documentation for timm. A big thanks to Aman Arora for his efforts creating timmdocs.

paperswithcode is a good resource for browsing the models within timm.

Train, Validation, Inference Scripts

The root folder of the repository contains reference train, validation, and inference scripts that work with the included models and other features of this repository. They are adaptable for other datasets and use cases with a little hacking. See documentation.

Awesome PyTorch Resources

One of the greatest assets of PyTorch is the community and their contributions. A few of my favourite resources that pair well with the models and components here are listed below.

Object Detection, Instance and Semantic Segmentation

Computer Vision / Image Augmentation

Knowledge Distillation

Metric Learning

Training / Frameworks



The code here is licensed Apache 2.0. I've taken care to make sure any third party code included or adapted has compatible (permissive) licenses such as MIT, BSD, etc. I've made an effort to avoid any GPL / LGPL conflicts. That said, it is your responsibility to ensure you comply with licenses here and conditions of any dependent licenses. Where applicable, I've linked the sources/references for various components in docstrings. If you think I've missed anything please create an issue.

Pretrained Weights

So far all of the pretrained weights available here are pretrained on ImageNet with a select few that have some additional pretraining (see extra note below). ImageNet was released for non-commercial research purposes only (https://image-net.org/download). It's not clear what the implications of that are for the use of pretrained weights from that dataset. Any models I have trained with ImageNet are done for research purposes and one should assume that the original dataset license applies to the weights. It's best to seek legal advice if you intend to use the pretrained weights in a commercial product.

Pretrained on more than ImageNet

Several weights included or references here were pretrained with proprietary datasets that I do not have access to. These include the Facebook WSL, SSL, SWSL ResNe(Xt) and the Google Noisy Student EfficientNet models. The Facebook models have an explicit non-commercial license (CC-BY-NC 4.0, https://github.com/facebookresearch/semi-supervised-ImageNet1K-models, https://github.com/facebookresearch/WSL-Images). The Google models do not appear to have any restriction beyond the Apache 2.0 license (and ImageNet concerns). In either case, you should contact Facebook or Google with any questions.



  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/rwightman/pytorch-image-models}}

Latest DOI

3 weeks, 2 days ago

Revolutionizing Database Interactions with Private LLM Technology

DB-GPT: Revolutionizing Database Interactions with Private LLM Technology

What is DB-GPT?

DB-GPT is an experimental open-source project that uses localized GPT large models to interact with your data and environment. With this solution, you can be assured that there is no risk of data leakage, and your data is 100% private and secure.


DB-GPT Youtube Video


Run on an RTX 4090 GPU.

Chat Excel

Chat Plugin

LLM Management

FastChat && vLLM


Chat Knowledge


Usage Tutorial


Currently, we have released multiple key features, which are listed below to demonstrate our current capabilities:

  • Private KBQA & data processing

    The DB-GPT project offers a range of features to enhance knowledge base construction and enable efficient storage and retrieval of both structured and unstructured data. These include built-in support for uploading multiple file formats, the ability to integrate plug-ins for custom data extraction, and unified vector storage and retrieval capabilities for managing large volumes of information.

  • Multiple data sources & visualization

    The DB-GPT project enables seamless natural language interaction with various data sources, including Excel, databases, and data warehouses. It facilitates effortless querying and retrieval of information from these sources, allowing users to engage in intuitive conversations and obtain insights. Additionally, DB-GPT supports the generation of analysis reports, providing users with valuable summaries and interpretations of the data.

  • Multi-Agents&Plugins

    It supports custom plug-ins to perform tasks, natively supports the Auto-GPT plug-in model, and the Agents protocol adopts the Agent Protocol standard.

  • Fine-tuning text2SQL

    An automated fine-tuning lightweight framework built around large language models, Text2SQL data sets, LoRA/QLoRA/Pturning, and other fine-tuning methods, making TextSQL fine-tuning as convenient as an assembly line. DB-GPT-Hub

  • Multi LLMs Support, Supports multiple large language models, currently supporting

    Massive model support, including dozens of large language models such as open source and API agents. Such as LLaMA/LLaMA2, Baichuan, ChatGLM, Wenxin, Tongyi, Zhipu, etc.


  • Privacy and security

    The privacy and security of data are ensured through various technologies, such as privatized large models and proxy desensitization.

  • Support Datasources

DataSource support Notes
PostgreSQL Yes
Spark Yes
DuckDB Yes
Sqlite Yes
ClickHouse Yes
Oracle No TODO
Redis No TODO
Doris No TODO
Couchbase No TODO
Elasticsearch No TODO
OceanBase No TODO
StarRocks No TODO


The architecture of the entire DB-GPT is shown.

The core capabilities mainly consist of the following parts:

  1. Multi-Models: Support multi-LLMs, such as LLaMA/LLaMA2、CodeLLaMA、ChatGLM, QWen、Vicuna and proxy model ChatGPT、Baichuan、tongyi、wenxin etc
  2. Knowledge-Based QA: You can perform high-quality intelligent Q&A based on local documents such as PDF, word, excel, and other data.
  3. Embedding: Unified data vector storage and indexing, Embed data as vectors and store them in vector databases, providing content similarity search.
  4. Multi-Datasources: Used to connect different modules and data sources to achieve data flow and interaction.
  5. Multi-Agents: Provides Agent and plugin mechanisms, allowing users to customize and enhance the system's behavior.
  6. Privacy & Secure: You can be assured that there is no risk of data leakage, and your data is 100% private and secure.
  7. Text2SQL: We enhance the Text-to-SQL performance by applying Supervised Fine-Tuning (SFT) on large language models



  • DB-GPT-Hub Text-to-SQL performance by applying Supervised Fine-Tuning (SFT) on large language models.
  • DB-GPT-Plugins DB-GPT Plugins Can run autogpt plugin directly
  • DB-GPT-Web ChatUI for DB-GPT


🌐 AutoDL Image

Language Switching

In the .env configuration file, modify the LANGUAGE parameter to switch to different languages. The default is English (Chinese: zh, English: en, other languages to be added later).


  • Please run black . before submitting the code. Contributing guidelines, how to contribute


KBQA RAG optimization

  • Multi Documents

    • PDF
    • Excel, CSV
    • Word
    • Text
    • MarkDown
    • Code
    • Images
  • RAG

  • Graph Database

    • Neo4j Graph
    • Nebula Graph
  • Multi-Vector Database

    • Chroma
    • Milvus
    • Weaviate
    • PGVector
    • Elasticsearch
    • ClickHouse
    • Faiss
  • Testing and Evaluation Capability Building

    • Knowledge QA datasets
    • Question collection [easy, medium, hard]:
    • Scoring mechanism
    • Testing and evaluation using Excel + DB datasets

Multi Datasource Support

  • Multi Datasource Support
    • MySQL
    • PostgreSQL
    • Spark
    • DuckDB
    • Sqlite
    • MSSQL
    • ClickHouse
    • Oracle
    • Redis
    • MongoDB
    • HBase
    • Doris
    • DB2
    • Couchbase
    • Elasticsearch
    • OceanBase
    • TiDB
    • StarRocks

Multi-Models And vLLM

Agents market and Plugins

  • multi-agents framework
  • custom plugin development
  • plugin market
  • Integration with CoT
  • Enrich plugin sample library
  • Support for AutoGPT protocol
  • Integration of multi-agents and visualization capabilities, defining LLM+Vis new standards

Cost and Observability

Text2SQL Finetune

  • support llms

    • LLaMA
    • LLaMA-2
    • BLOOM
    • BLOOMZ
    • Falcon
    • Baichuan
    • Baichuan2
    • InternLM
    • Qwen
    • XVERSE
    • ChatGLM2
  • SFT Accuracy

As of October 10, 2023, by fine-tuning an open-source model of 13 billion parameters using this project, the execution accuracy on the Spider evaluation dataset has surpassed that of GPT-4!

name Execution Accuracy reference
GPT-4 0.762 numbersstation-eval-res
ChatGPT 0.728 numbersstation-eval-res
CodeLlama-13b-Instruct-hf_lora 0.789 sft train by our this project,only used spider train dataset ,the same eval way in this project with lora SFT
CodeLlama-13b-Instruct-hf_qlora 0.774 sft train by our this project,only used spider train dataset ,the same eval way in this project with qlora and nf4,bit4 SFT
wizardcoder 0.610 text-to-sql-wizardcoder
CodeLlama-13b-Instruct-hf 0.556 eval in this project default param
llama2_13b_hf_lora_best 0.744 sft train by our this project,only used spider train dataset ,the same eval way in this project

More Information about Text2SQL finetune


The MIT License (MIT)

Contact Information

We are working on building a community, if you have any ideas for building the community, feel free to contact us.

3 weeks, 2 days ago

Bisheng is an open LLM devops platform for next generation AI applications.

欢迎来到 Bisheng

Bisheng 是什么



Bisheng 基于 Apache 2.0 License 协议发布,于 2023 年 8 月底正式开源。


  • 便捷:即使是业务人员,基于我们预置的应用模板,通过简单直观的表单填写方式快速搭建以大模型为核心的智能应用。
  • 灵活:对大模型技术有了解的人员,我们紧跟最前沿大模型技术生态提供数百种开发组件,基于可视化且自由的流程编排能力,可开发出任意类型的大模型应用,而不仅是简单的提示词工程。
  • 可靠与企业级:当前许多同类的开源项目仅适用于实验测试场景,缺少真正生产使用的企业级特性,包括:高并发下的高可用、应用运营及效果持续迭代优化、贴合真实业务场景的实用功能等,这些都是毕昇平台的差异化能力;另外,更直观的是,企业内的数据质量参差不齐,想要真正把所有数据利用起来,首先需要有完备的非结构化数据治理能力,而这是过去几年我们团队所积累的核心能力,在毕昇的demo环境中您可以通过相关组件直接接入这些能力,并且这些能力免费不限量使用。




  • 📃 合同审核报告生成
  • 🏦 信贷调查报告生成
  • 📈 招股书分析报告生成
  • 💼 智能投顾报告生成
  • 👀 文档摘要生成


  • 👩‍💻 用户手册问答
  • 👩🏻‍🔬 研报知识库问答
  • 🗄 规章制度问答
  • 💊 《中华药典》知识问答
  • 📊 股价数据库问答


  • 🎭 扮演面试官对话
  • 📍 小红书文案助手
  • 👩‍🎤 扮演外教对话
  • 👨‍🏫 简历优化助手


  • 📄 合同关键要素提取
  • 🏗️ 工程报告要素提取
  • 🗂️ 通用元数据提取
  • 🎫 卡证票据要素提取




启动 Bisheng

源码编译 Bisheng

Todo: 待补充

获取更多内容,请阅读 开发者文档


欢迎向 Bisheng 社区贡献你的代码。代码贡献流程或提交补丁等相关信息详见 代码贡献准则。 参考 社区仓库 了解社区管理准则并获取更多社区资源。

All Thanks To Our Contributors:

Bisheng 文档

获取更多有关安装、开发、部署和管理的指南,请查看 Bisheng 文档.


  • 欢迎加入 Slack 频道分享你的建议与问题。
  • 你也可以通过 FAQ 页面,查看常见问题及解答。
  • 你也可以加入 讨论组 发起问题和讨论。

关注 Bisheng 社交媒体:

  • Bisheng 技术交流微信群


DataElem Inc. 是 Bisheng 项目的幕后公司。我们正在 招聘 算法、开发和全栈工程师。欢迎加入我们,让我们携手构建下一代的智能应用开发平台。


Bisheng 采用了以下依赖库:

  • 感谢开源模型预估框架 Triton
  • 感谢开源LLM应用开发库 langchain
  • 感谢开源非结构化数据解析引擎 unstructured
  • 感谢开源langchain可视化工具 langflow

Star History

3 weeks, 3 days ago

Main Sigma Rule Repository

Sigma - Generic Signature Format for SIEM Systems

Welcome to the Sigma main rule repository. The place where detection engineers, threat hunters and all defensive security practitioners collaborate on detection rules. The repository offers more than 3000 detection rules of different type and aims to make reliable detections accessible to all at no cost.

Currently the repository offers three types of rules:

  • Generic Detection Rules - Are threat agnostic, their aim is to detect a behavior or an implementation of a technique or procedure that was, can or will be used by a potential threat actor.
  • Threat Hunting Rules - Are broader in scope and are meant to give the analyst a starting point to hunt for potential suspicious or malicious activity
  • Emerging Threat Rules - Are rules that cover specific threats, that are timely and relevant for certain periods of time. These threats include specific APT campaigns, exploitation of Zero-Day vulnerabilities, specific malware used during an attack,...etc.

Explore Sigma

To start exploring the Sigma ecosystem, please visit the official website sigmahq.io

What is Sigma

Sigma is a generic and open signature format that allows you to describe relevant log events in a straightforward manner. The rule format is very flexible, easy to write and applicable to any type of log file.

The main purpose of this project is to provide a structured form in which researchers or analysts can describe their once developed detection methods and make them shareable with others.

Sigma is for log files what Snort is for network traffic and YARA is for files.

Why Sigma

Today, everyone collects log data for analysis. People start working on their own, processing numerous white papers, blog posts and log analysis guidelines, extracting the necessary information and build their own searches and dashboard. Some of their searches and correlations are great and very useful but they lack a standardized format in which they can share their work with others.

Others provide excellent analyses, include IOCs and YARA rules to detect the malicious files and network connections, but have no way to describe a specific or generic detection method in log events. Sigma is meant to be an open standard in which such detection mechanisms can be defined, shared and collected in order to improve the detection capabilities for everyone.

🌟 Key Features

  • A continuously growing list of detection and hunting rules, peer reviewed by a community of professional Detection Engineers.
  • Vendor agnostic detection rules.
  • Easily shareable across communities and reports

🏗️ Rule Creation

To start writing Sigma rules please check the following guides:

🔎 Contributing & Making PRs

Please refer to the CONTRIBUTING guide for detailed instructions on how you can start contributing new rules.

📦 Rule Packages

You can download the latest rule packages from the release page and start leveraging Sigma rules today.

🧬 Rule Usage and Conversion

  • You can start converting Sigma rules today using Sigma CLI or sigconverter.io the GUI interface

  • To integrate Sigma rules in your own toolchain or products use pySigma.

🚨 Reporting False Positives or New Rule Ideas

If you find a false positive or would like to propose a new detection rule idea but do not have the time to create one, please create a new issue on the GitHub repository by selecting one of the available templates.

📚 Resources & Further Reading

Projects or Products that use or integrate Sigma rules

📜 Maintainers


This project would've never reached this hight without the help of the hundreds of contributors. Thanks to all past and present contributors for their help.


The content of this repository is released under the Detection Rule License (DRL) 1.1.

3 weeks, 3 days ago

Easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, ChatGLM3)

LLaMA Factory: Training and Evaluating Large Language Models with Minimal Effort

👋 Join our WeChat.

[ English | 中文 ]

LLaMA Board: A One-stop Web UI for Getting Started with LLaMA Factory

Launch LLaMA Board via CUDA_VISIBLE_DEVICES=0 python src/train_web.py. (multiple GPUs are not supported yet)

Here is an example of altering the self-cognition of an instruction-tuned language model within 10 minutes on a single GPU.



[23/10/21] We supported NEFTune trick for fine-tuning. Try --neft_alpha argument to activate NEFTune, e.g., --neft_alpha 5.

[23/09/27] We supported $S^2$-Attn proposed by LongLoRA for the LLaMA models. Try --shift_attn argument to enable shift short attention.

[23/09/23] We integrated MMLU, C-Eval and CMMLU benchmarks in this repo. See this example to evaluate your models.

[23/09/10] We supported using FlashAttention-2 for the LLaMA models. Try --flash_attn argument to enable FlashAttention-2 if you are using RTX4090, A100 or H100 GPUs.

[23/08/12] We supported RoPE scaling to extend the context length of the LLaMA models. Try --rope_scaling linear argument in training and --rope_scaling dynamic argument at inference to extrapolate the position embeddings.

[23/08/11] We supported DPO training for instruction-tuned models. See this example to train your models.

[23/07/31] We supported dataset streaming. Try --streaming and --max_steps 10000 arguments to load your dataset in streaming mode.

[23/07/29] We released two instruction-tuned 13B models at Hugging Face. See these Hugging Face Repos (LLaMA-2 / Baichuan) for details.

[23/07/18] We developed an all-in-one Web UI for training, evaluation and inference. Try train_web.py to fine-tune models in your Web browser. Thank @KanadeSiina and @codemayq for their efforts in the development.

[23/07/09] We released FastEdit ⚡🩹, an easy-to-use package for editing the factual knowledge of large language models efficiently. Please follow FastEdit if you are interested.

[23/06/29] We provided a reproducible example of training a chat model using instruction-following datasets, see Baichuan-7B-sft for details.

[23/06/22] We aligned the demo API with the OpenAI's format where you can insert the fine-tuned model in arbitrary ChatGPT-based applications.

[23/06/03] We supported quantized training and inference (aka QLoRA). Try --quantization_bit 4/8 argument to work with quantized models.

Supported Models

Model Model size Default module Template
LLaMA 7B/13B/33B/65B q_proj,v_proj -
LLaMA-2 7B/13B/70B q_proj,v_proj llama2
BLOOM 560M/1.1B/1.7B/3B/7.1B/176B query_key_value -
BLOOMZ 560M/1.1B/1.7B/3B/7.1B/176B query_key_value -
Falcon 7B/40B query_key_value -
Baichuan 7B/13B W_pack baichuan
Baichuan2 7B/13B W_pack baichuan2
InternLM 7B/20B q_proj,v_proj intern
Qwen 7B/14B c_attn chatml
ChatGLM3 6B query_key_value chatglm3
Phi-1.5 1.3B Wqkv -

[!NOTE] Default module is used for the --lora_target argument, you can use --lora_target all to specify all the available modules.

For the "base" models, the --template argument can be chosen from default, alpaca, vicuna etc. But make sure to use the corresponding template for the "chat" models.

Please refer to template.py for a full list of models we supported.

Supported Training Approaches

Approach Full-parameter Partial-parameter LoRA QLoRA
Supervised Fine-Tuning
Reward Modeling
PPO Training
DPO Training

[!NOTE] Use --quantization_bit 4/8 argument to enable QLoRA.

Provided Datasets

Pre-training datasets Supervised fine-tuning datasets Preference datasets

Please refer to data/README.md for details.

Some datasets require confirmation before using them, so we recommend logging in with your Hugging Face account using these commands.

pip install --upgrade huggingface_hub
huggingface-cli login


  • Python 3.8+ and PyTorch 1.13.1+
  • 🤗Transformers, Datasets, Accelerate, PEFT and TRL
  • sentencepiece, protobuf and tiktoken
  • fire, jieba, rouge-chinese and nltk (used at evaluation and predict)
  • gradio and matplotlib (used in web UI)
  • uvicorn, fastapi and sse-starlette (used in API)

And powerful GPUs!

Getting Started

Data Preparation (optional)

Please refer to data/README.md for checking the details about the format of dataset files. You can either use a single .json file or a dataset loading script with multiple files to create a custom dataset.

[!NOTE] Please update data/dataset_info.json to use your custom dataset. About the format of this file, please refer to data/README.md.

Dependence Installation (optional)

git clone https://github.com/hiyouga/LLaMA-Factory.git
conda create -n llama_factory python=3.10
conda activate llama_factory
cd LLaMA-Factory
pip install -r requirements.txt

If you want to enable the quantized LoRA (QLoRA) on the Windows platform, you will be required to install a pre-built version of bitsandbytes library, which supports CUDA 11.1 to 12.1.

pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl

Train on a single GPU

[!IMPORTANT] If you want to train models on multiple GPUs, please refer to Distributed Training.


CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
    --stage pt \
    --model_name_or_path path_to_llama_model \
    --do_train \
    --dataset wiki_demo \
    --finetuning_type lora \
    --lora_target q_proj,v_proj \
    --output_dir path_to_pt_checkpoint \
    --overwrite_cache \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 4 \
    --lr_scheduler_type cosine \
    --logging_steps 10 \
    --save_steps 1000 \
    --learning_rate 5e-5 \
    --num_train_epochs 3.0 \
    --plot_loss \

Supervised Fine-Tuning

CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
    --stage sft \
    --model_name_or_path path_to_llama_model \
    --do_train \
    --dataset alpaca_gpt4_en \
    --template default \
    --finetuning_type lora \
    --lora_target q_proj,v_proj \
    --output_dir path_to_sft_checkpoint \
    --overwrite_cache \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 4 \
    --lr_scheduler_type cosine \
    --logging_steps 10 \
    --save_steps 1000 \
    --learning_rate 5e-5 \
    --num_train_epochs 3.0 \
    --plot_loss \

Reward Modeling

CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
    --stage rm \
    --model_name_or_path path_to_llama_model \
    --do_train \
    --dataset comparison_gpt4_en \
    --template default \
    --finetuning_type lora \
    --lora_target q_proj,v_proj \
    --resume_lora_training False \
    --checkpoint_dir path_to_sft_checkpoint \
    --output_dir path_to_rm_checkpoint \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 4 \
    --lr_scheduler_type cosine \
    --logging_steps 10 \
    --save_steps 1000 \
    --learning_rate 1e-6 \
    --num_train_epochs 1.0 \
    --plot_loss \

PPO Training

CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
    --stage ppo \
    --model_name_or_path path_to_llama_model \
    --do_train \
    --dataset alpaca_gpt4_en \
    --template default \
    --finetuning_type lora \
    --lora_target q_proj,v_proj \
    --resume_lora_training False \
    --checkpoint_dir path_to_sft_checkpoint \
    --reward_model path_to_rm_checkpoint \
    --output_dir path_to_ppo_checkpoint \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 4 \
    --lr_scheduler_type cosine \
    --logging_steps 10 \
    --save_steps 1000 \
    --learning_rate 1e-5 \
    --num_train_epochs 1.0 \
    --plot_loss \

DPO Training

CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
    --stage dpo \
    --model_name_or_path path_to_llama_model \
    --do_train \
    --dataset comparison_gpt4_en \
    --template default \
    --finetuning_type lora \
    --lora_target q_proj,v_proj \
    --resume_lora_training False \
    --checkpoint_dir path_to_sft_checkpoint \
    --output_dir path_to_dpo_checkpoint \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 4 \
    --lr_scheduler_type cosine \
    --logging_steps 10 \
    --save_steps 1000 \
    --learning_rate 1e-5 \
    --num_train_epochs 1.0 \
    --plot_loss \

Distributed Training

Use Huggingface Accelerate

accelerate config # configure the environment
accelerate launch src/train_bash.py # arguments (same as above)
Example config for LoRA training
compute_environment: LOCAL_MACHINE
distributed_type: MULTI_GPU
downcast_bf16: 'no'
gpu_ids: all
machine_rank: 0
main_training_function: main
mixed_precision: fp16
num_machines: 1
num_processes: 4
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false

Use DeepSpeed

deepspeed --num_gpus 8 --master_port=9901 src/train_bash.py \
    --deepspeed ds_config.json \
    ... # arguments (same as above)
Example config for full-parameter training with DeepSpeed ZeRO-2
  "train_batch_size": "auto",
  "train_micro_batch_size_per_gpu": "auto",
  "gradient_accumulation_steps": "auto",
  "gradient_clipping": "auto",
  "zero_allow_untested_optimizer": true,
  "fp16": {
    "enabled": "auto",
    "loss_scale": 0,
    "initial_scale_power": 16,
    "loss_scale_window": 1000,
    "hysteresis": 2,
    "min_loss_scale": 1
  "zero_optimization": {
    "stage": 2,
    "allgather_partitions": true,
    "allgather_bucket_size": 5e8,
    "reduce_scatter": true,
    "reduce_bucket_size": 5e8,
    "overlap_comm": false,
    "contiguous_gradients": true

Export model

python src/export_model.py \
    --model_name_or_path path_to_llama_model \
    --template default \
    --finetuning_type lora \
    --checkpoint_dir path_to_checkpoint \
    --export_dir path_to_export

API Demo

python src/api_demo.py \
    --model_name_or_path path_to_llama_model \
    --template default \
    --finetuning_type lora \
    --checkpoint_dir path_to_checkpoint

[!NOTE] Visit http://localhost:8000/docs for API documentation.

CLI Demo

python src/cli_demo.py \
    --model_name_or_path path_to_llama_model \
    --template default \
    --finetuning_type lora \
    --checkpoint_dir path_to_checkpoint

Web Demo

python src/web_demo.py \
    --model_name_or_path path_to_llama_model \
    --template default \
    --finetuning_type lora \
    --checkpoint_dir path_to_checkpoint


CUDA_VISIBLE_DEVICES=0 python src/evaluate.py \
    --model_name_or_path path_to_llama_model \
    --finetuning_type lora \
    --checkpoint_dir path_to_checkpoint \
    --template vanilla \
    --task mmlu \
    --split test \
    --lang en \
    --n_shot 5 \
    --batch_size 4


CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
    --stage sft \
    --model_name_or_path path_to_llama_model \
    --do_predict \
    --dataset alpaca_gpt4_en \
    --template default \
    --finetuning_type lora \
    --checkpoint_dir path_to_checkpoint \
    --output_dir path_to_predict_result \
    --per_device_eval_batch_size 8 \
    --max_samples 100 \

[!NOTE] We recommend using --per_device_eval_batch_size=1 and --max_target_length 128 at 4/8-bit predict.

Projects using LLaMA Factory

  • StarWhisper: A large language model for Astronomy, based on ChatGLM2-6B and Qwen-14B.
  • DISC-LawLLM: A large language model specialized in Chinese legal domain, based on Baichuan-13B, is capable of retrieving and reasoning on legal knowledge.
  • Sunsimiao: A large language model specialized in Chinese medical domain, based on Baichuan-7B and ChatGLM-6B.
  • CareGPT: A series of large language models for Chinese medical domain, based on LLaMA2-7B and Baichuan-13B.


This repository is licensed under the Apache-2.0 License.

Please follow the model licenses to use the corresponding model weights: LLaMA / LLaMA-2 / BLOOM / Falcon / Baichuan / Baichuan2 / InternLM / Qwen / ChatGLM3 / Phi-1.5


If this work is helpful, please kindly cite as:

  title = {LLaMA Factory},
  author = {hiyouga},
  howpublished = {\url{https://github.com/hiyouga/LLaMA-Factory}},
  year = {2023}


This repo benefits from PEFT, QLoRA and FastChat. Thanks for their wonderful works.

Star History

3 weeks, 3 days ago


Use GitHub Copilot locally on your Macbook with one-click!

Demo Video


This video is not sped up or slowed down.


  1. First, open VS Code Settings and add the following to your settings.json file:
"github.copilot.advanced": {
    "debug.testOverrideProxyUrl": "http://localhost:5001",
    "debug.overrideProxyUrl": "http://localhost:5001"
  1. Create a virtualenv to run this Python process, install the requirements, and download the models.
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
# First setup run. This will download several models to your ~/models folder.
python app.py --setup 
  1. Run it!
python app.py

Enjoy your on-device Copilot!

Caveat FAQ

Is the code as good as GitHub Copilot?

For simple line completions yes. For simple function completions, mostly. For complex functions... maybe.

Is it as fast as GitHub Copilot?

On my Macbook Pro with an Apple M2 Max, the 7b models are roughly as fast. The 34b models are not. Please consider this repo a demonstration of a very inefficient implementation. I'm sure we can make it faster; please do submit a pull request if you'd like to help. For example, I think we need debouncer because sometimes llama.cpp/GGML isn't fast at interrupting itself when a newer request comes in.

Can this be packaged as a simple Mac app?

Yes!, I'm sure it can be, I just haven't had the time. Please do submit a pull request if you're into that sort of thing!

Should there be a meta-model that routes to a 1b for autocomplete, 7b for more complex autocomplete, and a 34b for program completion?

Hmm, that seems like an interesting idea.

OK, but in summary, is it good?

Only if your network is bad. I don't think it's competitive if you have fast Internet. But it sure is awesome on airplanes and while tethering!

3 weeks, 5 days ago

3D printer / slicing GUI built on top of the Uranium framework




Ultimaker Cura

State-of-the-art slicer app to prepare
your 3D models for your 3D printer.

With hundreds of settings & community-managed print profiles,
Ultimaker Cura is sure to lead your next project to a success.



3 weeks, 5 days ago

Efficient Streaming Language Models with Attention Sinks

Efficient Streaming Language Models with Attention Sinks

[paper] [slides][video]



We deploy LLMs for infinite-length inputs without sacrificing efficiency and performance.



Deploying Large Language Models (LLMs) in streaming applications such as multi-round dialogue, where long interactions are expected, is urgently needed but poses two major challenges. Firstly, during the decoding stage, caching previous tokens' Key and Value states (KV) consumes extensive memory. Secondly, popular LLMs cannot generalize to longer texts than the training sequence length. Window attention, where only the most recent KVs are cached, is a natural approach --- but we show that it fails when the text length surpasses the cache size. We observe an interesting phenomenon, namely attention sink, that keeping the KV of initial tokens will largely recover the performance of window attention. In this paper, we first demonstrate that the emergence of attention sink is due to the strong attention scores towards initial tokens as a ``sink'' even if they are not semantically important. Based on the above analysis, we introduce StreamingLLM, an efficient framework that enables LLMs trained with a finite length attention window to generalize to infinite sequence length without any fine-tuning. We show that StreamingLLM can enable Llama-2, MPT, Falcon, and Pythia to perform stable and efficient language modeling with up to 4 million tokens and more. In addition, we discover that adding a placeholder token as a dedicated attention sink during pre-training can further improve streaming deployment. In streaming settings, StreamingLLM outperforms the sliding window recomputation baseline by up to 22.2x speedup.


Environment Setup

conda create -yn streaming python=3.8
conda activate streaming

pip install torch torchvision torchaudio
pip install transformers==4.33.0 accelerate datasets evaluate wandb scikit-learn scipy sentencepiece

python setup.py develop

Run Streaming Llama Chatbot

CUDA_VISIBLE_DEVICES=0 python examples/run_streaming_llama.py  --enable_streaming


  1. What does "working on infinite-length inputs" imply for LLMs?

    Handling infinite-length text with LLMs presents challenges. Notably, storing all previous Key and Value (KV) states demands significant memory, and models might struggle to generate text beyond their training sequence length. StreamingLLM addresses this by retaining only the most recent tokens and attention sinks, discarding intermediate tokens. This enables the model to generate coherent text from recent tokens without a cache reset — a capability not seen in earlier methods.

  2. Is the context window of LLMs expanded?

    No. The context window remains unchanged. Only the most recent tokens and attention sinks are retained, discarding middle tokens. This means the model can only process the latest tokens. The context window remains constrained by its initial pre-training. For instance, if Llama-2 is pre-trained with a context window of 4096 tokens, then the maximum cache size for StreamingLLM on Llama-2 remains 4096.

  3. Can I input an extensive text, like a book, into StreamingLLM for summarization?

    While you can input a lengthy text, the model will only recognize the latest tokens. Thus, if a book is an input, StreamingLLM might only summarize the concluding paragraphs, which might not be very insightful. As emphasized earlier, we neither expand the LLMs' context window nor enhance their long-term memory. StreamingLLM's strength lies in generating fluent text from recent tokens without needing a cache refresh.

  4. What is the ideal use case for StreamingLLM?

    StreamingLLM is optimized for streaming applications, such as multi-round dialogues. It's ideal for scenarios where a model needs to operate continually without requiring extensive memory or dependency on past data. An example is a daily assistant based on LLMs. StreamingLLM would let the model function continuously, basing its responses on recent conversations without needing to refresh its cache. Earlier methods would either need a cache reset when the conversation length exceeded the training length (losing recent context) or recompute KV states from recent text history, which can be time-consuming.

  5. How does StreamingLLM relate to recent works on context extension?

    StreamingLLM is orthogonal to recent context extension methods and can be integrated with them. In StreamingLLM's context, "context extension" refers to the possibility of using a larger cache size to store more recent tokens. For a practical demonstration, refer to Figure 9 in our paper, where we implement StreamingLLM with models like LongChat-7B-v1.5-32K and Llama-2-7B-32K-Instruct.


We will release the code and data in the following order, please stay tuned!

  • Release core code of StreamingLLM, including Llama-2, MPT, Falcon, and Pythia.
  • Release perplexity evaluation code
  • Release Streaming Llama Chatbot demo.
  • Release StreamEval dataset and evaluation code.


If you find StreamingLLM useful or relevant to your project and research, please kindly cite our paper:

        title={Efficient Streaming Language Models with Attention Sinks},
        author={Xiao, Guangxuan and Tian, Yuandong and Chen, Beidi and Han, Song and Lewis, Mike},
4 weeks ago

A collective list of free APIs

Public APIs

A collective list of free APIs for use in software and web development

The Project
Contributing GuideAPI for this projectIssuesPull RequestsLicense
Alternative sites for the project (unofficials)
Free APIsDev ResourcesPublic APIs SiteApihouseCollective APIs



API Description Auth HTTPS CORS
AdoptAPet Resource to help get pets adopted apiKey Yes Yes
Axolotl Collection of axolotl pictures and facts No Yes No
Cat Facts Daily cat facts No Yes No
Cataas Cat as a service (cats pictures and gifs) No Yes No
Cats Pictures of cats from Tumblr apiKey Yes No
Dog Facts Random dog facts No Yes Yes
Dog Facts Random facts of Dogs No Yes Yes
Dogs Based on the Stanford Dogs Dataset No Yes Yes
eBird Retrieve recent or notable birding observations within a region apiKey Yes No
FishWatch Information and pictures about individual fish species No Yes Yes
HTTP Cat Cat for every HTTP Status No Yes Yes
HTTP Dog Dogs for every HTTP response status code No Yes Yes
IUCN IUCN Red List of Threatened Species apiKey No No
MeowFacts Get random cat facts No Yes No
Movebank Movement and Migration data of animals No Yes Yes
Petfinder Petfinder is dedicated to helping pets find homes, another resource to get pets adopted apiKey Yes Yes
PlaceBear Placeholder bear pictures No Yes Yes
PlaceDog Placeholder Dog pictures No Yes Yes
PlaceKitten Placeholder Kitten pictures No Yes Yes
RandomDog Random pictures of dogs No Yes Yes
RandomDuck Random pictures of ducks No Yes No
RandomFox Random pictures of foxes No Yes No
RescueGroups Adoption No Yes Unknown
Shibe.Online Random pictures of Shiba Inu, cats or birds No Yes Yes
The Dog A public service all about Dogs, free to use when making your fancy new App, Website or Service apiKey Yes No
xeno-canto Bird recordings No Yes Unknown
Zoo Animals Facts and pictures of zoo animals No Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
AniAPI Anime discovery, streaming & syncing with trackers OAuth Yes Yes
AniDB Anime Database apiKey No Unknown
AniList Anime discovery & tracking OAuth Yes Unknown
AnimeChan Anime quotes (over 10k+) No Yes No
AnimeFacts Anime Facts (over 100+) No Yes Yes
AnimeNewsNetwork Anime industry news No Yes Yes
Catboy Neko images, funny GIFs & more No Yes Yes
Danbooru Anime Thousands of anime artist database to find good anime art apiKey Yes Yes
Jikan Unofficial MyAnimeList API No Yes Yes
Kitsu Anime discovery platform OAuth Yes Yes
MangaDex Manga Database and Community apiKey Yes Unknown
Mangapi Translate manga pages from one language to another apiKey Yes Unknown
MyAnimeList Anime and Manga Database and Community OAuth Yes Unknown
NekosBest Neko Images & Anime roleplaying GIFs No Yes Yes
Shikimori Anime discovery, tracking, forum, rates OAuth Yes Unknown
Studio Ghibli Resources from Studio Ghibli films No Yes Yes
Trace Moe A useful tool to get the exact scene of an anime from a screenshot No Yes No
Waifu.im Get waifu pictures from an archive of over 4000 images and multiple tags No Yes Yes
Waifu.pics Image sharing platform for anime images No Yes No

⬆ Back to Index


API Description Auth HTTPS CORS
AbuseIPDB IP/domain/URL reputation apiKey Yes Unknown
AlienVault Open Threat Exchange (OTX) IP/domain/URL reputation apiKey Yes Unknown
CAPEsandbox Malware execution and analysis apiKey Yes Unknown
Google Safe Browsing Google Link/Domain Flagging apiKey Yes Unknown
MalDatabase Provide malware datasets and threat intelligence feeds apiKey Yes Unknown
MalShare Malware Archive / file sourcing apiKey Yes No
MalwareBazaar Collect and share malware samples apiKey Yes Unknown
Metacert Metacert Link Flagging apiKey Yes Unknown
NoPhishy Check links to see if they're known phishing attempts apiKey Yes Yes
Phisherman IP/domain/URL reputation apiKey Yes Unknown
Scanii Simple REST API that can scan submitted documents/files for the presence of threats apiKey Yes Yes
URLhaus Bulk queries and Download Malware Samples No Yes Yes
URLScan.io Scan and Analyse URLs apiKey Yes Unknown
VirusTotal VirusTotal File/URL Analysis apiKey Yes Unknown
Web of Trust IP/domain/URL reputation apiKey Yes Unknown

⬆ Back to Index

Art & Design

API Description Auth HTTPS CORS
Améthyste Generate images for Discord users apiKey Yes Unknown
Art Institute of Chicago Art No Yes Yes
Colormind Color scheme generator No No Unknown
ColourLovers Get various patterns, palettes and images No No Unknown
Cooper Hewitt Smithsonian Design Museum apiKey Yes Unknown
Dribbble Discover the world’s top designers & creatives OAuth Yes Unknown
EmojiHub Get emojis by categories and groups No Yes Yes
Europeana European Museum and Galleries content apiKey Yes Unknown
Harvard Art Museums Art apiKey No Unknown
Icon Horse Favicons for any website, with fallbacks No Yes Yes
Iconfinder Icons apiKey Yes Unknown
Icons8 Icons (find "search icon" hyperlink in page) No Yes Unknown
Lordicon Icons with predone Animations No Yes Yes
Metropolitan Museum of Art Met Museum of Art No Yes No
Noun Project Icons OAuth No Unknown
PHP-Noise Noise Background Image Generator No Yes Yes
Pixel Encounter SVG Icon Generator No Yes No
Rijksmuseum RijksMuseum Data apiKey Yes Unknown
Word Cloud Easily create word clouds apiKey Yes Unknown
xColors Generate & convert colors No Yes Yes

⬆ Back to Index

Authentication & Authorization

API Description Auth HTTPS CORS
Auth0 Easy to implement, adaptable authentication and authorization platform apiKey Yes Yes
GetOTP Implement OTP flow quickly apiKey Yes No
Micro User Service User management and authentication apiKey Yes No
MojoAuth Secure and modern passwordless authentication platform apiKey Yes Yes
SAWO Labs Simplify login and improve user experience by integrating passwordless authentication in your app apiKey Yes Yes
Stytch User infrastructure for modern applications apiKey Yes No
Warrant APIs for authorization and access control apiKey Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
Bitquery Onchain GraphQL APIs & DEX APIs apiKey Yes Yes
Chainlink Build hybrid smart contracts with Chainlink No Yes Unknown
Chainpoint Chainpoint is a global network for anchoring data to the Bitcoin blockchain No Yes Unknown
Covalent Multi-blockchain data aggregator platform apiKey Yes Unknown
Etherscan Ethereum explorer API apiKey Yes Yes
Helium Helium is a global, distributed network of Hotspots that create public, long-range wireless coverage No Yes Unknown
Nownodes Blockchain-as-a-service solution that provides high-quality connection via API apiKey Yes Unknown
Steem Blockchain-based blogging and social media website No No No
The Graph Indexing protocol for querying networks like Ethereum with GraphQL apiKey Yes Unknown
Walltime To retrieve Walltime's market info No Yes Unknown
Watchdata Provide simple and reliable API access to Ethereum blockchain apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
A Bíblia Digital Do not worry about managing the multiple versions of the Bible apiKey Yes No
Bhagavad Gita Open Source Shrimad Bhagavad Gita API including 21+ authors translation in Sanskrit/English/Hindi apiKey Yes Yes
Bhagavad Gita Bhagavad Gita text OAuth Yes Yes
Bhagavad Gita telugu Bhagavad Gita API in telugu and odia languages No Yes Yes
Bible-api Free Bible API with multiple languages No Yes Yes
British National Bibliography Books No No Unknown
Crossref Metadata Search Books & Articles Metadata No Yes Unknown
Ganjoor Classic Persian poetry works including access to related manuscripts, recitations and music tracks OAuth Yes Yes
Google Books Books OAuth Yes Unknown
GurbaniNow Fast and Accurate Gurbani RESTful API No Yes Unknown
Gutendex Web-API for fetching data from Project Gutenberg Books Library No Yes Unknown
Open Library Books, book covers and related data No Yes No
Penguin Publishing Books, book covers and related data No Yes Yes
PoetryDB Enables you to get instant data from our vast poetry collection No Yes Yes
Quran RESTful Quran API with multiple languages No Yes Yes
Quran Cloud A RESTful Quran API to retrieve an Ayah, Surah, Juz or the entire Holy Quran No Yes Yes
Quran-api Free Quran API Service with 90+ different languages and 400+ translations No Yes Yes
Rig Veda Gods and poets, their categories, and the verse meters, with the mandal and sukta number No Yes Unknown
The Bible Everything you need from the Bible in one discoverable place apiKey Yes Unknown
Thirukkural 1330 Thirukkural poems and explanation in Tamil and English No Yes Yes
Vedic Society Descriptions of all nouns (names, places, animals, things) from vedic literature No Yes Unknown
Wizard World Get information from the Harry Potter universe No Yes Yes
Wolne Lektury API for obtaining information about e-books available on the WolneLektury.pl website No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Apache Superset API to manage your BI dashboards and data sources on Superset apiKey Yes Yes
Charity Search Non-profit charity data apiKey No Unknown
Clearbit Logo Search for company logos and embed them in your projects apiKey Yes Unknown
Domainsdb.info Registered Domain Names Search No Yes No
Freelancer Hire freelancers to get work done OAuth Yes Unknown
Gmail Flexible, RESTful access to the user's inbox OAuth Yes Unknown
Google Analytics Collect, configure and analyze your data to reach the right audience OAuth Yes Unknown
Instatus Post to and update maintenance and incidents on your status page through an HTTP REST API apiKey Yes Unknown
Mailchimp Send marketing campaigns and transactional mails apiKey Yes Unknown
mailjet Marketing email can be sent and mail templates made in MJML or HTML can be sent using API apiKey Yes Unknown
markerapi Trademark Search No No Unknown
ORB Intelligence Company lookup apiKey Yes Unknown
Redash Access your queries and dashboards on Redash apiKey Yes Yes
Smartsheet Allows you to programmatically access and Smartsheet data and account information OAuth Yes No
Square Easy way to take payments, manage refunds, and help customers checkout online OAuth Yes Unknown
SwiftKanban Kanban software, Visualize Work, Increase Organizations Lead Time, Throughput & Productivity apiKey Yes Unknown
Tenders in Hungary Get data for procurements in Hungary in JSON format No Yes Unknown
Tenders in Poland Get data for procurements in Poland in JSON format No Yes Unknown
Tenders in Romania Get data for procurements in Romania in JSON format No Yes Unknown
Tenders in Spain Get data for procurements in Spain in JSON format No Yes Unknown
Tenders in Ukraine Get data for procurements in Ukraine in JSON format No Yes Unknown
Tomba email finder Email Finder for B2B sales and email marketing and email verifier apiKey Yes Yes
Trello Boards, lists and cards to help you organize and prioritize your projects OAuth Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Abstract Public Holidays Data on national, regional, and religious holidays via API apiKey Yes Yes
Calendarific Worldwide Holidays apiKey Yes Unknown
Checkiday - National Holiday API Industry-leading Holiday API. Over 5,000 holidays and thousands of descriptions. Trusted by the World’s leading companies apiKey Yes Unknown
Church Calendar Catholic liturgical calendar No No Unknown
Czech Namedays Calendar Lookup for a name and returns nameday date No No Unknown
Festivo Public Holidays Fastest and most advanced public holiday and observance service on the market apiKey Yes Yes
Google Calendar Display, create and modify Google calendar events OAuth Yes Unknown
Hebrew Calendar Convert between Gregorian and Hebrew, fetch Shabbat and Holiday times, etc No No Unknown
Holidays Historical data regarding holidays apiKey Yes Unknown
LectServe Protestant liturgical calendar No No Unknown
Nager.Date Public holidays for more than 90 countries No Yes No
Namedays Calendar Provides namedays for multiple countries No Yes Yes
Non-Working Days Database of ICS files for non working days No Yes Unknown
Non-Working Days Simple REST API for checking working, non-working or short days for Russia, CIS, USA and other No Yes Yes
Russian Calendar Check if a date is a Russian holiday or not No Yes No
UK Bank Holidays Bank holidays in England and Wales, Scotland and Northern Ireland No Yes Unknown

⬆ Back to Index

Cloud Storage & File Sharing

API Description Auth HTTPS CORS
AnonFiles Upload and share your files anonymously No Yes Unknown
BayFiles Upload and share your files No Yes Unknown
Box File Sharing and Storage OAuth Yes Unknown
ddownload File Sharing and Storage apiKey Yes Unknown
Dropbox File Sharing and Storage OAuth Yes Unknown
File.io Super simple file sharing, convenient, anonymous and secure No Yes Unknown
Filestack Filestack File Uploader & File Upload API apiKey Yes Unknown
GoFile Unlimited size file uploads for free apiKey Yes Unknown
Google Drive File Sharing and Storage OAuth Yes Unknown
Gyazo Save & Share screen captures instantly apiKey Yes Unknown
Imgbb Simple and quick private image sharing apiKey Yes Unknown
OneDrive File Sharing and Storage OAuth Yes Unknown
Pantry Free JSON storage for small projects No Yes Yes
Pastebin Plain Text Storage apiKey Yes Unknown
Pinata IPFS Pinning Services API apiKey Yes Unknown
Quip File Sharing and Storage for groups apiKey Yes Yes
Storj Decentralized Open-Source Cloud Storage apiKey Yes Unknown
The Null Pointer No-bullshit file hosting and URL shortening service No Yes Unknown
Web3 Storage File Sharing and Storage for Free with 1TB Space apiKey Yes Yes

⬆ Back to Index

Continuous Integration

API Description Auth HTTPS CORS
Azure DevOps Health Resource health helps you diagnose and get support when an Azure issue impacts your resources apiKey No No
Bitrise Build tool and processes integrations to create efficient development pipelines apiKey Yes Unknown
Buddy The fastest continuous integration and continuous delivery platform OAuth Yes Unknown
CircleCI Automate the software development process using continuous integration and continuous delivery apiKey Yes Unknown
Codeship Codeship is a Continuous Integration Platform in the cloud apiKey Yes Unknown
Travis CI Sync your GitHub projects with Travis CI to test your code in minutes apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
0x API for querying token and pool stats across various liquidity pools No Yes Yes
1inch API for querying decentralize exchange No Yes Unknown
Alchemy Ethereum Ethereum Node-as-a-Service Provider apiKey Yes Yes
apilayer coinlayer Real-time Crypto Currency Exchange Rates apiKey Yes Unknown
Binance Exchange for Trading Cryptocurrencies based in China apiKey Yes Unknown
Bitcambio Get the list of all traded assets in the exchange No Yes Unknown
BitcoinAverage Digital Asset Price Data for the blockchain industry apiKey Yes Unknown
BitcoinCharts Financial and Technical Data related to the Bitcoin Network No Yes Unknown
Bitfinex Cryptocurrency Trading Platform apiKey Yes Unknown
Bitmex Real-Time Cryptocurrency derivatives trading platform based in Hong Kong apiKey Yes Unknown
Bittrex Next Generation Crypto Trading Platform apiKey Yes Unknown
Block Bitcoin Payment, Wallet & Transaction Data apiKey Yes Unknown
Blockchain Bitcoin Payment, Wallet & Transaction Data apiKey Yes Unknown
blockfrost Cardano Interaction with the Cardano mainnet and several testnets apiKey Yes Unknown
Brave NewCoin Real-time and historic crypto data from more than 200+ exchanges apiKey Yes Unknown
BtcTurk Real-time cryptocurrency data, graphs and API that allows buy&sell apiKey Yes Yes
Bybit Cryptocurrency data feed and algorithmic trading apiKey Yes Unknown
CoinAPI All Currency Exchanges integrate under a single api apiKey Yes No
Coinbase Bitcoin, Bitcoin Cash, Litecoin and Ethereum Prices apiKey Yes Unknown
Coinbase Pro Cryptocurrency Trading Platform apiKey Yes Unknown
CoinCap Real time Cryptocurrency prices through a RESTful API No Yes Unknown
CoinDCX Cryptocurrency Trading Platform apiKey Yes Unknown
CoinDesk CoinDesk's Bitcoin Price Index (BPI) in multiple currencies No Yes Unknown
CoinGecko Cryptocurrency Price, Market, and Developer/Social Data No Yes Yes
Coinigy Interacting with Coinigy Accounts and Exchange Directly apiKey Yes Unknown
Coinlib Crypto Currency Prices apiKey Yes Unknown
Coinlore Cryptocurrencies prices, volume and more No Yes Unknown
CoinMarketCap Cryptocurrencies Prices apiKey Yes Unknown
Coinpaprika Cryptocurrencies prices, volume and more No Yes Yes
CoinRanking Live Cryptocurrency data apiKey Yes Unknown
Coinremitter Cryptocurrencies Payment & Prices apiKey Yes Unknown
CoinStats Crypto Tracker No Yes Unknown
CryptAPI Cryptocurrency Payment Processor No Yes Unknown
CryptingUp Cryptocurrency data No Yes Unknown
CryptoCompare Cryptocurrencies Comparison No Yes Unknown
CryptoMarket Cryptocurrencies Trading platform apiKey Yes Yes
Cryptonator Cryptocurrencies Exchange Rates No Yes Unknown
dYdX Decentralized cryptocurrency exchange apiKey Yes Unknown
Ethplorer Ethereum tokens, balances, addresses, history of transactions, contracts, and custom structures apiKey Yes Unknown
EXMO Cryptocurrencies exchange based in UK apiKey Yes Unknown
FTX Complete REST, websocket, and FTX APIs to suit your algorithmic trading needs apiKey Yes Yes
Gateio API provides spot, margin and futures trading operations apiKey Yes Unknown
Gemini Cryptocurrencies Exchange No Yes Unknown
Hirak Exchange Rates Exchange rates between 162 currency & 300 crypto currency update each 5 min, accurate, no limits apiKey Yes Unknown
Huobi Seychelles based cryptocurrency exchange apiKey Yes Unknown
icy.tools GraphQL based NFT API apiKey Yes Unknown
Indodax Trade your Bitcoin and other assets with rupiah apiKey Yes Unknown
INFURA Ethereum Interaction with the Ethereum mainnet and several testnets apiKey Yes Yes
Kraken Cryptocurrencies Exchange apiKey Yes Unknown
KuCoin Cryptocurrency Trading Platform apiKey Yes Unknown
Localbitcoins P2P platform to buy and sell Bitcoins No Yes Unknown
Mempool Bitcoin API Service focusing on the transaction fee No Yes No
MercadoBitcoin Brazilian Cryptocurrency Information No Yes Unknown
Messari Provides API endpoints for thousands of crypto assets No Yes Unknown
Nexchange Automated cryptocurrency exchange service No No Yes
Nomics Historical and realtime cryptocurrency prices and market data apiKey Yes Yes
NovaDax NovaDAX API to access all market data, trading management endpoints apiKey Yes Unknown
OKEx Cryptocurrency exchange based in Seychelles apiKey Yes Unknown
Poloniex US based digital asset exchange apiKey Yes Unknown
Solana JSON RPC Provides various endpoints to interact with the Solana Blockchain No Yes Unknown
Technical Analysis Cryptocurrency prices and technical analysis apiKey Yes No
VALR Cryptocurrency Exchange based in South Africa apiKey Yes Unknown
WorldCoinIndex Cryptocurrencies Prices apiKey Yes Unknown
ZMOK Ethereum JSON RPC API and Web3 provider No Yes Unknown

⬆ Back to Index

Currency Exchange

API Description Auth HTTPS CORS
1Forge Forex currency market data apiKey Yes Unknown
Amdoren Free currency API with over 150 currencies apiKey Yes Unknown
apilayer fixer.io Exchange rates and currency conversion apiKey No Unknown
Bank of Russia Exchange rates and currency conversion No Yes Unknown
Currency-api Free Currency Exchange Rates API with 150+ Currencies & No Rate Limits No Yes Yes
CurrencyFreaks Provides current and historical currency exchange rates with free plan 1K requests/month apiKey Yes Yes
Currencylayer Exchange rates and currency conversion apiKey Yes Unknown
CurrencyScoop Real-time and historical currency rates JSON API apiKey Yes Yes
Czech National Bank A collection of exchange rates No Yes Unknown
Economia.Awesome Portuguese free currency prices and conversion with no rate limits No Yes Unknown
ExchangeRate-API Free currency conversion apiKey Yes Yes
Exchangerate.host Free foreign exchange & crypto rates API No Yes Unknown
Exchangeratesapi.io Exchange rates with currency conversion apiKey Yes Yes
Frankfurter Exchange rates, currency conversion and time series No Yes Yes
FreeForexAPI Real-time foreign exchange rates for major currency pairs No Yes No
National Bank of Poland A collection of currency exchange rates (data in XML and JSON) No Yes Yes
VATComply.com Exchange rates, geolocation and VAT number validation No Yes Yes

⬆ Back to Index

Data Validation

API Description Auth HTTPS CORS
Lob.com US Address Verification apiKey Yes Unknown
Postman Echo Test api server to receive and return value from HTTP method No Yes Unknown
PurgoMalum Content validator against profanity & obscenity No No Unknown
US Autocomplete Enter address data quickly with real-time address suggestions apiKey Yes Yes
US Extract Extract postal addresses from any text including emails apiKey Yes Yes
US Street Address Validate and append data for any US postal address apiKey Yes Yes
vatlayer VAT number validation apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
24 Pull Requests Project to promote open source collaboration during December No Yes Yes
Abstract Screenshot Take programmatic screenshots of web pages from any website apiKey Yes Yes
Agify.io Estimates the age from a first name No Yes Yes
API Grátis Multiples services and public APIs No Yes Unknown
ApicAgent Extract device details from user-agent string No Yes Yes
ApiFlash Chrome based screenshot API for developers apiKey Yes Unknown
apilayer userstack Secure User-Agent String Lookup JSON API OAuth Yes Unknown
APIs.guru Wikipedia for Web APIs, OpenAPI/Swagger specs for public APIs No Yes Unknown
Azure DevOps The Azure DevOps basic components of a REST API request/response pair apiKey Yes Unknown
Base Building quick backends apiKey Yes Yes
Beeceptor Build a mock Rest API endpoint in seconds No Yes Yes
Bitbucket Bitbucket API OAuth Yes Unknown
Blague.xyz La plus grande API de Blagues FR/The biggest FR jokes API apiKey Yes Yes
Blitapp Schedule screenshots of web pages and sync them to your cloud apiKey Yes Unknown
Blynk-Cloud Control IoT Devices from Blynk IoT Cloud apiKey No Unknown
Bored Find random activities to fight boredom No Yes Unknown
Brainshop.ai Make A Free A.I Brain apiKey Yes Yes
Browshot Easily make screenshots of web pages in any screen size, as any device apiKey Yes Yes
CDNJS Library info on CDNJS No Yes Unknown
Changelogs.md Structured changelog metadata from open source projects No Yes Unknown
Ciprand Secure random string generator No Yes No
Cloudflare Trace Get IP Address, Timestamp, User Agent, Country Code, IATA, HTTP Version, TLS/SSL Version & More No Yes Yes
Codex Online Compiler for Various Languages No Yes Unknown
Contentful Images Used to retrieve and apply transformations to images apiKey Yes Yes
CORS Proxy Get around the dreaded CORS error by using this proxy as a middle man No Yes Yes
CountAPI Free and simple counting service. You can use it to track page hits and specific events No Yes Yes
Databricks Service to manage your databricks account,clusters, notebooks, jobs and workspaces apiKey Yes Yes
DigitalOcean Status Status of all DigitalOcean services No Yes Unknown
Docker Hub Interact with Docker Hub apiKey Yes Yes
DomainDb Info Domain name search to find all domains containing particular words/phrases/etc No Yes Unknown
ExtendsClass JSON Storage A simple JSON store API No Yes Yes
GeekFlare Provide numerous capabilities for important testing and monitoring methods for websites apiKey Yes Unknown
Genderize.io Estimates a gender from a first name No Yes Yes
GETPing Trigger an email notification with a simple GET request apiKey Yes Unknown
Ghost Get Published content into your Website, App or other embedded media apiKey Yes Yes
GitHub Make use of GitHub repositories, code and user info programmatically OAuth Yes Yes
Gitlab Automate GitLab interaction programmatically OAuth Yes Unknown
Gitter Chat for Developers OAuth Yes Unknown
Glitterly Image generation API apiKey Yes Yes
Google Docs API to read, write, and format Google Docs documents OAuth Yes Unknown
Google Firebase Google's mobile application development platform that helps build, improve, and grow app apiKey Yes Yes
Google Fonts Metadata for all families served by Google Fonts apiKey Yes Unknown
Google Keep API to read, write, and format Google Keep notes OAuth Yes Unknown
Google Sheets API to read, write, and format Google Sheets data OAuth Yes Unknown
Google Slides API to read, write, and format Google Slides presentations OAuth Yes Unknown
Gorest Online REST API for Testing and Prototyping OAuth Yes Unknown
Hasura GraphQL and REST API Engine with built in Authorization apiKey Yes Yes
Heroku REST API to programmatically create apps, provision add-ons and perform other task on Heroku OAuth Yes Yes
host-t.com Basic DNS query via HTTP GET request No Yes No
Host.io Domains Data API for Developers apiKey Yes Yes
HTTP2.Pro Test endpoints for client and server HTTP/2 protocol support No Yes Unknown
Httpbin A Simple HTTP Request & Response Service No Yes Yes
Httpbin Cloudflare A Simple HTTP Request & Response Service with HTTP/3 Support by Cloudflare No Yes Yes
Hunter API for domain search, professional email finder, author finder and email verifier apiKey Yes Unknown
IBM Text to Speech Convert text to speech apiKey Yes Yes
Icanhazepoch Get Epoch time No Yes Yes
Icanhazip IP Address API No Yes Yes
IFTTT IFTTT Connect API No Yes Unknown
Image-Charts Generate charts, QR codes and graph images No Yes Yes
import.io Retrieve structured data from a website or RSS feed apiKey Yes Unknown
ip-fast.com IP address, country and city No Yes Yes
IP2WHOIS Information Lookup WHOIS domain name lookup apiKey Yes Unknown
ipfind.io Geographic location of an IP address or any domain name along with some other useful information apiKey Yes Yes
IPify A simple IP Address API No Yes Unknown
IPinfo Another simple IP Address API No Yes Unknown
jsDelivr Package info and download stats on jsDelivr CDN No Yes Yes
JSON 2 JSONP Convert JSON to JSONP (on-the-fly) for easy cross-domain data requests using client-side JavaScript No Yes Unknown
JSONbin.io Free JSON storage service. Ideal for small scale Web apps, Websites and Mobile apps apiKey Yes Yes
Kroki Creates diagrams from textual descriptions No Yes Yes
License-API Unofficial REST API for choosealicense.com No Yes No
Logs.to Generate logs apiKey Yes Unknown
Lua Decompiler Online Lua 5.1 Decompiler No Yes Yes
MAC address vendor lookup Retrieve vendor details and other information regarding a given MAC address or an OUI apiKey Yes Yes
Micro DB Simple database service apiKey Yes Unknown
MicroENV Fake Rest API for developers No Yes Unknown
Mocky Mock user defined test JSON for REST API endpoints No Yes Yes
MY IP Get IP address information No Yes Unknown
Nationalize.io Estimate the nationality of a first name No Yes Yes
Netlify Netlify is a hosting service for the programmable web OAuth Yes Unknown
NetworkCalc Network calculators, including subnets, DNS, binary, and security tools No Yes Yes
npm Registry Query information about your favorite Node.js libraries programatically No Yes Unknown
OneSignal Self-serve customer engagement solution for Push Notifications, Email, SMS & In-App apiKey Yes Unknown
Open Page Rank API for calculating and comparing metrics of different websites using Page Rank algorithm apiKey Yes Unknown
OpenAPIHub The All-in-one API Platform X-Mashape-Key Yes Unknown
OpenGraphr Really simple API to retrieve Open Graph data from an URL apiKey Yes Unknown
oyyi API for Fake Data, image/video conversion, optimization, pdf optimization and thumbnail generation No Yes Yes
PageCDN Public API for javascript, css and font libraries on PageCDN apiKey Yes Yes
Postman Tool for testing APIs apiKey Yes Unknown
ProxyCrawl Scraping and crawling anticaptcha service apiKey Yes Unknown
ProxyKingdom Rotating Proxy API that produces a working proxy on every request apiKey Yes Yes
Pusher Beams Push notifications for Android & iOS apiKey Yes Unknown
QR code Create an easy to read QR code and URL shortener No Yes Yes
QR code Generate and decode / read QR code graphics No Yes Unknown
Qrcode Monkey Integrate custom and unique looking QR codes into your system or workflow No Yes Unknown
QuickChart Generate chart and graph images No Yes Yes
Random Stuff Can be used to get AI Response, jokes, memes, and much more at lightning-fast speed apiKey Yes Yes
Rejax Reverse AJAX service to notify clients apiKey Yes No
ReqRes A hosted REST-API ready to respond to your AJAX requests No Yes Unknown
RSS feed to JSON Returns RSS feed in JSON format using feed URL No Yes Yes
SavePage.io A free, RESTful API used to screenshot any desktop, or mobile website apiKey Yes Yes
ScrapeNinja Scraping API with Chrome fingerprint and residential proxies apiKey Yes Unknown
ScraperApi Easily build scalable web scrapers apiKey Yes Unknown
scraperBox Undetectable web scraping API apiKey Yes Yes
scrapestack Real-time, Scalable Proxy & Web Scraping REST API apiKey Yes Unknown
ScrapingAnt Headless Chrome scraping with a simple API apiKey Yes Unknown
ScrapingDog Proxy API for Web scraping apiKey Yes Unknown
ScreenshotAPI.net Create pixel-perfect website screenshots apiKey Yes Yes
Serialif Color Color conversion, complementary, grayscale and contrasted text No Yes No
serpstack Real-Time & Accurate Google Search Results API apiKey Yes Yes
Sheetsu Easy google sheets integration apiKey Yes Unknown
Sonar Project Sonar DNS Enumeration API No Yes Yes
SonarQube SonarQube REST APIs to detect bugs, code smells & security vulnerabilities OAuth Yes Unknown
StackExchange Q&A forum for developers OAuth Yes Unknown
Statically A free CDN for developers No Yes Yes
Supportivekoala Autogenerate images with template apiKey Yes Yes
Tyk Api and service management platform apiKey Yes Yes
Wandbox Code compiler supporting 35+ languages mentioned at wandbox.org No Yes Unknown
WebScraping.AI Web Scraping API with built-in proxies and JS rendering apiKey Yes Yes
ZenRows Web Scraping API that bypasses anti-bot solutions while offering JS rendering, and rotating proxies apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Chinese Character Web Chinese character definitions and pronunciations No No No
Chinese Text Project Online open-access digital library for pre-modern Chinese texts No Yes Unknown
Collins Bilingual Dictionary and Thesaurus Data apiKey Yes Unknown
Free Dictionary Definitions, phonetics, pronounciations, parts of speech, examples, synonyms No Yes Unknown
Indonesia Dictionary Indonesia dictionary many words No Yes Unknown
Lingua Robot Word definitions, pronunciations, synonyms, antonyms and others apiKey Yes Yes
Merriam-Webster Dictionary and Thesaurus Data apiKey Yes Unknown
OwlBot Definitions with example sentence and photo if available apiKey Yes Yes
Oxford Dictionary Data apiKey Yes No
Synonyms Synonyms, thesaurus and antonyms information for any given word apiKey Yes Unknown
Wiktionary Collaborative dictionary data No Yes Yes
Wordnik Dictionary Data apiKey Yes Unknown
Words Definitions and synonyms for more than 150,000 words apiKey Yes Unknown

⬆ Back to Index

Documents & Productivity

API Description Auth HTTPS CORS
Airtable Integrate with Airtable apiKey Yes Unknown
Api2Convert Online File Conversion API apiKey Yes Unknown
apilayer pdflayer HTML/URL to PDF apiKey Yes Unknown
Asana Programmatic access to all data in your asana system apiKey Yes Yes
ClickUp ClickUp is a robust, cloud-based project management tool for boosting productivity OAuth Yes Unknown
Clockify Clockify's REST-based API can be used to push/pull data to/from it & integrate it with other systems apiKey Yes Unknown
CloudConvert Online file converter for audio, video, document, ebook, archive, image, spreadsheet, presentation apiKey Yes Unknown
Cloudmersive Document and Data Conversion HTML/URL to PDF/PNG, Office documents to PDF, image conversion apiKey Yes Yes
Code::Stats Automatic time tracking for programmers apiKey Yes No
CraftMyPDF Generate PDF documents from templates with a drop-and-drop editor and a simple API apiKey Yes No
Flowdash Automate business workflows apiKey Yes Unknown
Html2PDF HTML/URL to PDF apiKey Yes Unknown
iLovePDF Convert, merge, split, extract text and add page numbers for PDFs. Free for 250 documents/month apiKey Yes Yes
JIRA JIRA is a proprietary issue tracking product that allows bug tracking and agile project management OAuth Yes Unknown
Mattermost An open source platform for developer collaboration OAuth Yes Unknown
Mercury Web parser apiKey Yes Unknown
Monday Programmatically access and update data inside a monday.com account apiKey Yes Unknown
Notion Integrate with Notion OAuth Yes Unknown
PandaDoc DocGen and eSignatures API apiKey Yes No
Pocket Bookmarking service OAuth Yes Unknown
Podio File sharing and productivity OAuth Yes Unknown
PrexView Data from XML or JSON to PDF, HTML or Image apiKey Yes Unknown
Restpack Provides screenshot, HTML to PDF and content extraction APIs apiKey Yes Unknown
Todoist Todo Lists OAuth Yes Unknown
Smart Image Enhancement API Performs image upscaling by adding detail to images through multiple super-resolution algorithms apiKey Yes Unknown
Vector Express v2.0 Free vector file converting API No Yes No
WakaTime Automated time tracking leaderboards for programmers No Yes Unknown
Zube Full stack project management OAuth Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Abstract Email Validation Validate email addresses for deliverability and spam apiKey Yes Yes
apilayer mailboxlayer Email address validation apiKey Yes Unknown
Cloudmersive Validate Validate email addresses, phone numbers, VAT numbers and domain names apiKey Yes Yes
Disify Validate and detect disposable and temporary email addresses No Yes Yes
DropMail GraphQL API for creating and managing ephemeral e-mail inboxes No Yes Unknown
EVA Validate email addresses No Yes Yes
Guerrilla Mail Disposable temporary Email addresses No Yes Unknown
ImprovMX API for free email forwarding service apiKey Yes Unknown
Kickbox Email verification API No Yes Yes
mail.gw 10 Minute Mail No Yes Yes
mail.tm Temporary Email Service No Yes Yes
MailboxValidator Validate email address to improve deliverability apiKey Yes Unknown
MailCheck.ai Prevent users to sign up with temporary email addresses No Yes Unknown
Mailtrap A service for the safe testing of emails sent from the development and staging environments apiKey Yes Unknown
Sendgrid A cloud-based SMTP provider that allows you to send emails without having to maintain email servers apiKey Yes Unknown
Sendinblue A service that provides solutions relating to marketing and/or transactional email and/or SMS apiKey Yes Unknown
Verifier Verifies that a given email is real apiKey Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
chucknorris.io JSON API for hand curated Chuck Norris jokes No Yes Unknown
Corporate Buzz Words REST API for Corporate Buzz Words No Yes Yes
Excuser Get random excuses for various situations No Yes Unknown
Fun Fact A simple HTTPS api that can randomly select and return a fact from the FFA database No Yes Yes
Imgflip Gets an array of popular memes No Yes Unknown
Meme Maker REST API for create your own meme No Yes Unknown
NaMoMemes Memes on Narendra Modi No Yes Unknown
Random Useless Facts Get useless, but true facts No Yes Unknown
Techy JSON and Plaintext API for tech-savvy sounding phrases No Yes Unknown
Yo Momma Jokes REST API for Yo Momma Jokes No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
BreezoMeter Pollen Daily Forecast pollen conditions data for a specific location apiKey Yes Unknown
Carbon Interface API to calculate carbon (C02) emissions estimates for common C02 emitting activities apiKey Yes Yes
Climatiq Calculate the environmental footprint created by a broad range of emission-generating activities apiKey Yes Yes
Cloverly API calculates the impact of common carbon-intensive activities in real time apiKey Yes Unknown
CO2 Offset API calculates and validates the carbon footprint No Yes Unknown
Danish data service Energi Open energy data from Energinet to society No Yes Unknown
GrünstromIndex Green Power Index for Germany (Grünstromindex/GSI) No No Yes
IQAir Air quality and weather data apiKey Yes Unknown
Luchtmeetnet Predicted and actual air quality components for The Netherlands (RIVM) No Yes Unknown
National Grid ESO Open data from Great Britain’s Electricity System Operator No Yes Unknown
OpenAQ Open air quality data apiKey Yes Unknown
PM2.5 Open Data Portal Open low-cost PM2.5 sensor data No Yes Unknown
PM25.in Air quality of China apiKey No Unknown
PVWatts Energy production photovoltaic (PV) energy systems apiKey Yes Unknown
Srp Energy Hourly usage energy report for Srp customers apiKey Yes No
UK Carbon Intensity The Official Carbon Intensity API for Great Britain developed by National Grid No Yes Unknown
Website Carbon API to estimate the carbon footprint of loading web pages No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Eventbrite Find events OAuth Yes Unknown
SeatGeek Search events, venues and performers apiKey Yes Unknown
Ticketmaster Search events, attractions, or venues apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Abstract VAT Validation Validate VAT numbers and calculate VAT rates apiKey Yes Yes
Aletheia Insider trading data, earnings call analysis, financial statements, and more apiKey Yes Yes
Alpaca Realtime and historical market data on all US equities and ETFs apiKey Yes Yes
Alpha Vantage Realtime and historical stock data apiKey Yes Unknown
apilayer marketstack Real-Time, Intraday & Historical Market Data API apiKey Yes Unknown
Banco do Brasil All Banco do Brasil financial transaction APIs OAuth Yes Yes
Bank Data API Instant IBAN and SWIFT number validation across the globe apiKey Yes Unknown
Billplz Payment platform apiKey Yes Unknown
Binlist Public access to a database of IIN/BIN information No Yes Unknown
Boleto.Cloud A api to generate boletos in Brazil apiKey Yes Unknown
Citi All Citigroup account and statement data APIs apiKey Yes Unknown
Econdb Global macroeconomic data No Yes Yes
Fed Treasury U.S. Department of the Treasury Data No Yes Unknown
Finage Finage is a stock, currency, cryptocurrency, indices, and ETFs real-time & historical data provider apiKey Yes Unknown
Financial Modeling Prep Realtime and historical stock data apiKey Yes Unknown
Finnhub Real-Time RESTful APIs and Websocket for Stocks, Currencies, and Crypto apiKey Yes Unknown
FRED Economic data from the Federal Reserve Bank of St. Louis apiKey Yes Yes
Front Accounting APIs Front accounting is multilingual and multicurrency software for small businesses OAuth Yes Yes
Hotstoks Stock market data powered by SQL apiKey Yes Yes
IEX Cloud Realtime & Historical Stock and Market Data apiKey Yes Yes
IG Spreadbetting and CFD Market Data apiKey Yes Unknown
Indian Mutual Fund Get complete history of India Mutual Funds Data No Yes Unknown
Intrinio A wide selection of financial data feeds apiKey Yes Unknown
Klarna Klarna payment and shopping service apiKey Yes Unknown
MercadoPago Mercado Pago API reference - all the information you need to develop your integrations apiKey Yes Unknown
Mono Connect with users’ bank accounts and access transaction data in Africa apiKey Yes Unknown
Moov The Moov API makes it simple for platforms to send, receive, and store money apiKey Yes Unknown
Nordigen Connect to bank accounts using official bank APIs and get raw transaction data apiKey Yes Unknown
OpenFIGI Equity, index, futures, options symbology from Bloomberg LP apiKey Yes Yes
Plaid Connect with user's bank accounts and access transaction data apiKey Yes Unknown
Polygon Historical stock market data apiKey Yes Unknown
Portfolio Optimizer Portfolio analysis and optimization No Yes Yes
Razorpay IFSC Indian Financial Systems Code (Bank Branch Codes) No Yes Unknown
Real Time Finance Websocket API to access realtime stock data apiKey No Unknown
SEC EDGAR Data API to access annual reports of public US companies No Yes Yes
SmartAPI Gain access to set of and create end-to-end broking services apiKey Yes Unknown
StockData Real-Time, Intraday & Historical Market Data, News and Sentiment API apiKey Yes Yes
Styvio Realtime and historical stock data and current stock sentiment apiKey Yes Unknown
Tax Data API Instant VAT number and tax validation across the globe apiKey Yes Unkown
Tradier US equity/option market data (delayed, intraday, historical) OAuth Yes Yes
Twelve Data Stock market data (real-time & historical) apiKey Yes Unknown
WallstreetBets WallstreetBets Stock Comments Sentiment Analysis No Yes Unknown
Yahoo Finance Real time low latency Yahoo Finance API for stock market, crypto currencies, and currency exchange apiKey Yes Yes
YNAB Budgeting & Planning OAuth Yes Yes
Zoho Books Online accounting software, built for your business OAuth Yes Unknown

⬆ Back to Index

Food & Drink

API Description Auth HTTPS CORS
BaconMockup Resizable bacon placeholder images No Yes Yes
Chomp Data about various grocery products and foods apiKey Yes Unknown
Coffee Random pictures of coffee No Yes Unknown
Edamam nutrition Nutrition Analysis apiKey Yes Unknown
Edamam recipes Recipe Search apiKey Yes Unknown
Foodish Random pictures of food dishes No Yes Yes
Fruityvice Data about all kinds of fruit No Yes Unknown
Kroger Supermarket Data apiKey Yes Unknown
LCBO Alcohol apiKey Yes Unknown
Open Brewery DB Breweries, Cideries and Craft Beer Bottle Shops No Yes Yes
Open Food Facts Food Products Database No Yes Unknown
PunkAPI Brewdog Beer Recipes No Yes Unknown
Rustybeer Beer brewing tools No Yes No
Spoonacular Recipes, Food Products, and Meal Planning apiKey Yes Unknown
Systembolaget Govornment owned liqour store in Sweden apiKey Yes Unknown
TacoFancy Community-driven taco database No No Unknown
Tasty API to query data about recipe, plan, ingredients apiKey Yes Unknown
The Report of the Week Food & Drink Reviews No Yes Unknown
TheCocktailDB Cocktail Recipes apiKey Yes Yes
TheMealDB Meal Recipes apiKey Yes Yes
Untappd Social beer sharing OAuth Yes Unknown
What's on the menu? NYPL human-transcribed historical menu collection apiKey No Unknown
WhiskyHunter Past online whisky auctions statistical data No Yes Unknown
Zestful Parse recipe ingredients apiKey Yes Yes

⬆ Back to Index

Games & Comics

API Description Auth HTTPS CORS
Age of Empires II Get information about Age of Empires II resources No Yes No
AmiiboAPI Nintendo Amiibo Information No Yes Yes
Animal Crossing: New Horizons API for critters, fossils, art, music, furniture and villagers No Yes Unknown
Autochess VNG Rest Api for Autochess VNG No Yes Yes
Barter.VG Provides information about Game, DLC, Bundles, Giveaways, Trading No Yes Yes
Battle.net Diablo III, Hearthstone, StarCraft II and World of Warcraft game data APIs OAuth Yes Yes
Board Game Geek Board games, RPG and videogames No Yes No
Brawl Stars Brawl Stars Game Information apiKey Yes Unknown
Bugsnax Get information about Bugsnax No Yes Yes
CheapShark Steam/PC Game Prices and Deals No Yes Yes
Chess.com Chess.com read-only REST API No Yes Unknown
Chuck Norris Database Jokes No No Unknown
Clash of Clans Clash of Clans Game Information apiKey Yes Unknown
Clash Royale Clash Royale Game Information apiKey Yes Unknown
Comic Vine Comics No Yes Unknown
Crafatar API for Minecraft skins and faces No Yes Yes
Cross Universe Cross Universe Card Data No Yes Yes
Deck of Cards Deck of Cards No No Unknown
Destiny The Game Bungie Platform API apiKey Yes Unknown
Digimon Information Provides information about digimon creatures No Yes Unknown
Digimon TCG Search for Digimon cards in digimoncard.io No Yes Unknown
Disney Information of Disney characters No Yes Yes
Dota 2 Provides information about Player stats , Match stats, Rankings for Dota 2 apiKey Yes Unknown
Dungeons and Dragons Reference for 5th edition spells, classes, monsters, and more No No No
Dungeons and Dragons (Alternate) Includes all monsters and spells from the SRD (System Reference Document) as well as a search API No Yes Yes
Eve Online Third-Party Developer Documentation OAuth Yes Unknown
FFXIV Collect Final Fantasy XIV data on collectables No Yes Yes
FIFA Ultimate Team FIFA Ultimate Team items API No Yes Unknown
Final Fantasy XIV Final Fantasy XIV Game data API No Yes Yes
Fortnite Fortnite Stats apiKey Yes Unknown
Forza Show random image of car from Forza No Yes Unknown
FreeToGame Free-To-Play Games Database No Yes Yes
Fun Facts Random Fun Facts No Yes Yes
FunTranslations Translate Text into funny languages No Yes Yes
GamerPower Game Giveaways Tracker No Yes Yes
GDBrowser Easy way to use the Geometry Dash Servers No Yes Unknown
Geek-Jokes Fetch a random geeky/programming related joke for use in all sorts of applications No Yes Yes
Genshin Impact Genshin Impact game data No Yes Yes
Giant Bomb Video Games apiKey Yes Unknown
GraphQL Pokemon GraphQL powered Pokemon API. Supports generations 1 through 8 No Yes Yes
Guild Wars 2 Guild Wars 2 Game Information apiKey Yes Unknown
GW2Spidy GW2Spidy API, Items data on the Guild Wars 2 Trade Market No Yes Unknown
Halo Halo 5 and Halo Wars 2 Information apiKey Yes Unknown
Hearthstone Hearthstone Cards Information X-Mashape-Key Yes Unknown
Humble Bundle Humble Bundle's current bundles apiKey Yes Unknown
Humor Humor, Jokes, and Memes apiKey Yes Unknown
Hypixel Hypixel player stats apiKey Yes Unknown
Hyrule Compendium Data on all interactive items from The Legend of Zelda: BOTW No Yes Unknown
Hytale Hytale blog posts and jobs No Yes Unknown
IGDB.com Video Game Database apiKey Yes Unknown
JokeAPI Programming, Miscellaneous and Dark Jokes No Yes Yes
Jokes One Joke of the day and large category of jokes accessible via REST API apiKey Yes Yes
Jservice Jeopardy Question Database No No Unknown
Lichess Access to all data of users, games, puzzles and etc on Lichess OAuth Yes Unknown
Magic The Gathering Magic The Gathering Game Information No No Unknown
Mario Kart Tour API for Drivers, Karts, Gliders and Courses OAuth Yes Unknown
Marvel Marvel Comics apiKey Yes Unknown
Minecraft Server Status API to get Information about a Minecraft Server No Yes No
MMO Games MMO Games Database, News and Giveaways No Yes No
mod.io Cross Platform Mod API apiKey Yes Unknown
Mojang Mojang / Minecraft API apiKey Yes Unknown
Monster Hunter World Monster Hunter World data No Yes Yes
Open Trivia Trivia Questions No Yes Unknown
PandaScore E-sports games and results apiKey Yes Unknown
Path of Exile Path of Exile Game Information OAuth Yes Unknown
PlayerDB Query Minecraft, Steam and XBox Accounts No Yes Unknown
Pokéapi Pokémon Information No Yes Unknown
PokéAPI (GraphQL) The Unofficial GraphQL for PokeAPI No Yes Yes
Pokémon TCG Pokémon TCG Information No Yes Unknown
Psychonauts Psychonauts World Characters Information and PSI Powers No Yes Yes
PUBG Access in-game PUBG data apiKey Yes Yes
Puyo Nexus Puyo Puyo information from Puyo Nexus Wiki No Yes Yes
quizapi.io Access to various kind of quiz questions apiKey Yes Yes
Raider Provides detailed character and guild rankings for Raiding and Mythic+ content in World of Warcraft No Yes Unknown
RAWG.io 500,000+ games for 50 platforms including mobiles apiKey Yes Unknown
Rick and Morty All the Rick and Morty information, including images No Yes Yes
Riot Games League of Legends Game Information apiKey Yes Unknown
RPS 101 Rock, Paper, Scissors with 101 objects No Yes Yes
RuneScape RuneScape and OSRS RPGs information No Yes No
Sakura CardCaptor Sakura CardCaptor Cards Information No Yes Unknown
Scryfall Magic: The Gathering database No Yes Yes
SpaceTradersAPI A playable inter-galactic space trading MMOAPI OAuth Yes Yes
Steam Steam Web API documentation apiKey Yes No
Steam Internal Steam Web API documentation No Yes No
SuperHeroes All SuperHeroes and Villains data from all universes under a single API apiKey Yes Unknown
TCGdex Multi languages Pokémon TCG Information No Yes Yes
Tebex Tebex API for information about game purchases X-Mashape-Key Yes No
TETR.IO TETR.IO Tetra Channel API No Yes Unknown
Tronald Dump The dumbest things Donald Trump has ever said No Yes Unknown
Universalis Final Fantasy XIV market board data No Yes Yes
Valorant (non-official) An extensive API containing data of most Valorant in-game items, assets and more No Yes Unknown
Warface (non-official) Official API proxy with better data structure and more features No Yes No
Wargaming.net Wargaming.net info and stats apiKey Yes No
When is next MCU film Upcoming MCU film information No Yes Unknown
xkcd Retrieve xkcd comics as JSON No Yes No
Yu-Gi-Oh! Yu-Gi-Oh! TCG Information No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Abstract IP Geolocation Geolocate website visitors from their IPs apiKey Yes Yes
Actinia Grass GIS Actinia is an open source REST API for geographical data that uses GRASS GIS apiKey Yes Unknown
administrative-divisons-db Get all administrative divisions of a country No Yes Yes
adresse.data.gouv.fr Address database of France, geocoding and reverse No Yes Unknown
Airtel IP IP Geolocation API. Collecting data from multiple sources No Yes Unknown
Apiip Get location information by IP address apiKey Yes Yes
apilayer ipstack Locate and identify website visitors by IP address apiKey Yes Unknown
Battuta A (country/region/city) in-cascade location API apiKey No Unknown
BigDataCloud Provides fast and accurate IP geolocation APIs along with security checks and confidence area apiKey Yes Unknown
Bing Maps Create/customize digital maps based on Bing Maps data apiKey Yes Unknown
bng2latlong Convert British OSGB36 easting and northing (British National Grid) to WGS84 latitude and longitude No Yes Yes
Cartes.io Create maps and markers for anything No Yes Unknown
Cep.la Brazil RESTful API to find information about streets, zip codes, neighborhoods, cities and states No No Unknown
CitySDK Open APIs for select European cities No Yes Unknown
Country Get your visitor's country from their IP No Yes Yes
CountryStateCity World countries, states, regions, provinces, cities & towns in JSON, SQL, XML, YAML, & CSV format apiKey Yes Yes
Ducks Unlimited API explorer that gives a query URL with a JSON response of locations and cities No Yes No
FreeGeoIP Free geo ip information, no registration required. 15k/hour rate limit No Yes Yes
GeoApi French geographical data No Yes Unknown
Geoapify Forward and reverse geocoding, address autocomplete apiKey Yes Yes
Geocod.io Address geocoding / reverse geocoding in bulk apiKey Yes Unknown
Geocode.xyz Provides worldwide forward/reverse geocoding, batch geocoding and geoparsing No Yes Unknown
Geocodify.com Worldwide geocoding, geoparsing and autocomplete for addresses apiKey Yes Yes
Geodata.gov.gr Open geospatial data and API service for Greece No Yes Unknown
GeoDataSource Geocoding of city name by using latitude and longitude coordinates apiKey Yes Unknown
GeoDB Cities Get global city, region, and country data apiKey Yes Unknown
GeographQL A Country, State, and City GraphQL API No Yes Yes
GeoJS IP geolocation with ChatOps integration No Yes Yes
Geokeo Geokeo geocoding service- with 2500 free api requests daily No Yes Yes
GeoNames Place names and other geographical data No No Unknown
geoPlugin IP geolocation and currency conversion No Yes Yes
Google Earth Engine A cloud-based platform for planetary-scale environmental data analysis apiKey Yes Unknown
Google Maps Create/customize digital maps based on Google Maps data apiKey Yes Unknown
Graph Countries Country-related data like currencies, languages, flags, regions+subregions and bordering countries No Yes Unknown
HelloSalut Get hello translation following user language No Yes Unknown
HERE Maps Create/customize digital maps based on HERE Maps data apiKey Yes Unknown
Hirak IP to Country Ip to location with country code, currency code & currency name, fast response, unlimited requests apiKey Yes Unknown
Hong Kong GeoData Store API for accessing geo-data of Hong Kong No Yes Unknown
IBGE Aggregate services of IBGE (Brazilian Institute of Geography and Statistics) No Yes Unknown
IP 2 Country Map an IP to a country No Yes Unknown
IP Address Details Find geolocation with ip address No Yes Unknown
IP Vigilante Free IP Geolocation API No Yes Unknown
ip-api Find location with IP address or domain No No Unknown
IP2Location IP geolocation web service to get more than 55 parameters apiKey Yes Unknown
IP2Proxy Detect proxy and VPN using IP address apiKey Yes Unknown
ipapi.co Find IP address location information No Yes Yes
ipapi.com Real-time Geolocation & Reverse IP Lookup REST API apiKey Yes Unknown
IPGEO Unlimited free IP Address API with useful information No Yes Unknown
ipgeolocation IP Geolocation AP with free plan 30k requests per month apiKey Yes Yes
IPInfoDB Free Geolocation tools and APIs for country, region, city and time zone lookup by IP address apiKey Yes Unknown
ipstack Locate and identify website visitors by IP address apiKey Yes Unknown
Kakao Maps Kakao Maps provide multiple APIs for Korean maps apiKey Yes Unknown
keycdn IP Location Finder Get the IP geolocation data through the simple REST API. All the responses are JSON encoded apiKey Yes Unknown
LocationIQ Provides forward/reverse geocoding and batch geocoding apiKey Yes Yes
Longdo Map Interactive map with detailed places and information portal in Thailand apiKey Yes Yes
Mapbox Create/customize beautiful digital maps apiKey Yes Unknown
MapQuest To access tools and resources to map the world apiKey Yes No
Mexico Mexico RESTful zip codes API No Yes Unknown
Nominatim Provides worldwide forward / reverse geocoding No Yes Yes
One Map, Singapore Singapore Land Authority REST API services for Singapore addresses apiKey Yes Unknown
OnWater Determine if a lat/lon is on water or land No Yes Unknown
Open Topo Data Elevation and ocean depth for a latitude and longitude No Yes No
OpenCage Forward and reverse geocoding using open data apiKey Yes Yes
openrouteservice.org Directions, POIs, isochrones, geocoding (+reverse), elevation, and more apiKey Yes Unknown
OpenStreetMap Navigation, geolocation and geographical data OAuth No Unknown
Pinball Map A crowdsourced map of public pinball machines No Yes Yes
positionstack Forward & Reverse Batch Geocoding REST API apiKey Yes Unknown
Postali Mexico Zip Codes API No Yes Yes
PostcodeData.nl Provide geolocation data based on postcode for Dutch addresses No No Unknown
Postcodes.io Postcode lookup & Geolocation for the UK No Yes Yes
Queimadas INPE Access to heat focus data (probable wildfire) No Yes Unknown
REST Countries Get information about countries via a RESTful API No Yes Yes
RoadGoat Cities Cities content & photos API apiKey Yes No
Rwanda Locations Rwanda Provences, Districts, Cities, Capital City, Sector, cells, villages and streets No Yes Unknown
SLF German city, country, river, database No Yes Yes
SpotSense Add location based interactions to your mobile app apiKey Yes Unknown
Telize Telize offers location information from any IP address apiKey Yes Yes
TomTom Maps, Directions, Places and Traffic APIs apiKey Yes Yes
Uebermaps Discover and share maps with friends apiKey Yes Unknown
US ZipCode Validate and append data for any US ZipCode apiKey Yes Yes
Utah AGRC Utah Web API for geocoding Utah addresses apiKey Yes Unknown
ViaCep Brazil RESTful zip codes API No Yes Unknown
What3Words Three words as rememberable and unique coordinates worldwide apiKey Yes Unknown
Yandex.Maps Geocoder Use geocoding to get an object's coordinates from its address apiKey Yes Unknown
ZipCodeAPI US zip code distance, radius and location API apiKey Yes Unknown
Zippopotam.us Get information about place such as country, city, state, etc No No Unknown
Ziptastic Get the country, state, and city of any US zip-code No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Bank Negara Malaysia Open Data Malaysia Central Bank Open Data No Yes Unknown
BCLaws Access to the laws of British Columbia No No Unknown
Brazil Community driven API for Brazil Public Data No Yes Yes
Brazil Central Bank Open Data Brazil Central Bank Open Data No Yes Unknown
Brazil Receita WS Consult companies by CNPJ for Brazilian companies No Yes Unknown
Brazilian Chamber of Deputies Open Data Provides legislative information in Apis XML and JSON, as well as files in various formats No Yes No
Census.gov The US Census Bureau provides various APIs and data sets on demographics and businesses No Yes Unknown
City, Berlin Berlin(DE) City Open Data No Yes Unknown
City, Gdańsk Gdańsk (PL) City Open Data No Yes Unknown
City, Gdynia Gdynia (PL) City Open Data No No Unknown
City, Helsinki Helsinki(FI) City Open Data No Yes Unknown
City, Lviv Lviv(UA) City Open Data No Yes Unknown
City, Nantes Open Data Nantes(FR) City Open Data apiKey Yes Unknown
City, New York Open Data New York (US) City Open Data No Yes Unknown
City, Prague Open Data Prague(CZ) City Open Data No No Unknown
City, Toronto Open Data Toronto (CA) City Open Data No Yes Yes
Code.gov The primary platform for Open Source and code sharing for the U.S. Federal Government apiKey Yes Unknown
Colorado Information Marketplace Colorado State Government Open Data No Yes Unknown
Data USA US Public Data No Yes Unknown
Data.gov US Government Data apiKey Yes Unknown
Data.parliament.uk Contains live datasets including information about petitions, bills, MP votes, attendance and more No No Unknown
Deutscher Bundestag DIP This API provides read access to DIP entities (e.g. activities, persons, printed material) apiKey Yes Unknown
District of Columbia Open Data Contains D.C. government public datasets, including crime, GIS, financial data, and so on No Yes Unknown
EPA Web services and data sets from the US Environmental Protection Agency No Yes Unknown
FBI Wanted Access information on the FBI Wanted program No Yes Unknown
FEC Information on campaign donations in federal elections apiKey Yes Unknown
Federal Register The Daily Journal of the United States Government No Yes Unknown
Food Standards Agency UK food hygiene rating data API No No Unknown
Gazette Data, UK UK official public record API OAuth Yes Unknown
Gun Policy International firearm injury prevention and policy apiKey Yes Unknown
INEI Peruvian Statistical Government Open Data No No Unknown
Interpol Red Notices Access and search Interpol Red Notices No Yes Unknown
Istanbul (İBB) Open Data Data sets from the İstanbul Metropolitan Municipality (İBB) No Yes Unknown
National Park Service, US Data from the US National Park Service apiKey Yes Yes
Open Government, ACT Australian Capital Territory Open Data No Yes Unknown
Open Government, Argentina Argentina Government Open Data No Yes Unknown
Open Government, Australia Australian Government Open Data No Yes Unknown
Open Government, Austria Austria Government Open Data No Yes Unknown
Open Government, Belgium Belgium Government Open Data No Yes Unknown
Open Government, Canada Canadian Government Open Data No No Unknown
Open Government, Colombia Colombia Government Open Data No No Unknown
Open Government, Cyprus Cyprus Government Open Data No Yes Unknown
Open Government, Czech Republic Czech Republic Government Open Data No Yes Unknown
Open Government, Denmark Denmark Government Open Data No Yes Unknown
Open Government, Estonia Estonia Government Open Data apiKey Yes Unknown
Open Government, Finland Finland Government Open Data No Yes Unknown
Open Government, France French Government Open Data apiKey Yes Unknown
Open Government, Germany Germany Government Open Data No Yes Unknown
Open Government, Greece Greece Government Open Data OAuth Yes Unknown
Open Government, India Indian Government Open Data apiKey Yes Unknown
Open Government, Ireland Ireland Government Open Data No Yes Unknown
Open Government, Italy Italy Government Open Data No Yes Unknown
Open Government, Korea Korea Government Open Data apiKey Yes Unknown
Open Government, Lithuania Lithuania Government Open Data No Yes Unknown
Open Government, Luxembourg Luxembourgish Government Open Data apiKey Yes Unknown
Open Government, Mexico Mexican Statistical Government Open Data No Yes Unknown
Open Government, Mexico Mexico Government Open Data No Yes Unknown
Open Government, Netherlands Netherlands Government Open Data No Yes Unknown
Open Government, New South Wales New South Wales Government Open Data apiKey Yes Unknown
Open Government, New Zealand New Zealand Government Open Data No Yes Unknown
Open Government, Norway Norwegian Government Open Data No Yes Yes
Open Government, Peru Peru Government Open Data No Yes Unknown
Open Government, Poland Poland Government Open Data No Yes Yes
Open Government, Portugal Portugal Government Open Data No Yes Yes
Open Government, Queensland Government Queensland Government Open Data No Yes Unknown
Open Government, Romania Romania Government Open Data No No Unknown
Open Government, Saudi Arabia Saudi Arabia Government Open Data No Yes Unknown
Open Government, Singapore Singapore Government Open Data No Yes Unknown
Open Government, Slovakia Slovakia Government Open Data No Yes Unknown
Open Government, Slovenia Slovenia Government Open Data No Yes No
Open Government, South Australian Government South Australian Government Open Data No Yes Unknown
Open Government, Spain Spain Government Open Data No Yes Unknown
Open Government, Sweden Sweden Government Open Data No Yes Unknown
Open Government, Switzerland Switzerland Government Open Data No Yes Unknown
Open Government, Taiwan Taiwan Government Open Data No Yes Unknown
Open Government, Thailand Thailand Government Open Data apiKey Yes Unknown
Open Government, UK UK Government Open Data No Yes Unknown
Open Government, USA United States Government Open Data No Yes Unknown
Open Government, Victoria State Government Victoria State Government Open Data No Yes Unknown
Open Government, West Australia West Australia Open Data No Yes Unknown
PRC Exam Schedule Unofficial Philippine Professional Regulation Commission's examination schedule No Yes Yes
Represent by Open North Find Canadian Government Representatives No Yes Unknown
UK Companies House UK Companies House Data from the UK government OAuth Yes Unknown
US Presidential Election Data by TogaTech Basic candidate data and live electoral vote counts for top two parties in US presidential election No Yes No
USA.gov Authoritative information on U.S. programs, events, services and more apiKey Yes Unknown
USAspending.gov US federal spending data No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
CMS.gov Access to the data from the CMS - medicare.gov apiKey Yes Unknown
Coronavirus HTTP API for Latest Covid-19 Data No Yes Unknown
Coronavirus in the UK UK Government coronavirus data, including deaths and cases by region No Yes Unknown
Covid Tracking Project Covid-19 data for the US No Yes No
Covid-19 Covid 19 spread, infection and recovery No Yes Yes
Covid-19 Covid 19 cases, deaths and recovery per country No Yes Yes
Covid-19 Datenhub Maps, datasets, applications and more in the context of COVID-19 No Yes Unknown
Covid-19 Government Response Government measures tracker to fight against the Covid-19 pandemic No Yes Yes
Covid-19 India Covid 19 statistics state and district wise about cases, vaccinations, recovery within India No Yes Unknown
Covid-19 JHU CSSE Open-source API for exploring Covid19 cases based on JHU CSSE No Yes Yes
Covid-19 Live Data Global and countrywise data of Covid 19 daily Summary, confirmed cases, recovered and deaths No Yes Yes
Covid-19 Philippines Unofficial Covid-19 Web API for Philippines from data collected by DOH No Yes Yes
COVID-19 Tracker Canada Details on Covid-19 cases across Canada No Yes Unknown
COVID-19 Tracker Sri Lanka Provides situation of the COVID-19 patients reported in Sri Lanka No Yes Unknown
COVID-ID Indonesian government Covid data per province No Yes Yes
Dataflow Kit COVID-19 COVID-19 live statistics into sites per hour No Yes Unknown
FoodData Central National Nutrient Database for Standard Reference apiKey Yes Unknown
Healthcare.gov Educational content about the US Health Insurance Marketplace No Yes Unknown
Humanitarian Data Exchange Humanitarian Data Exchange (HDX) is open platform for sharing data across crises and organisations No Yes Unknown
Infermedica NLP based symptom checker and patient triage API for health diagnosis from text apiKey Yes Yes
LAPIS SARS-CoV-2 genomic sequences from public sources No Yes Yes
Lexigram NLP that extracts mentions of clinical concepts from text, gives access to clinical ontology apiKey Yes Unknown
Makeup Makeup Information No No Unknown
MyVaccination Vaccination data for Malaysia No Yes Unknown
NPPES National Plan & Provider Enumeration System, info on healthcare providers registered in US No Yes Unknown
Nutritionix Worlds largest verified nutrition database apiKey Yes Unknown
Open Data NHS Scotland Medical reference data and statistics by Public Health Scotland No Yes Unknown
Open Disease API for Current cases and more stuff about COVID-19 and Influenza No Yes Yes
openFDA Public FDA data about drugs, devices and foods apiKey Yes Unknown
Orion Health Medical platform which allows the development of applications for different healthcare scenarios OAuth Yes Unknown
Quarantine Coronavirus API with free COVID-19 live updates No Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
Adzuna Job board aggregator apiKey Yes Unknown
Arbeitnow API for Job board aggregator in Europe / Remote No Yes Yes
Arbeitsamt API for the "Arbeitsamt", which is a german Job board aggregator OAuth Yes Unknown
Careerjet Job search engine apiKey No Unknown
DevITjobs UK Jobs with GraphQL No Yes Yes
Findwork Job board apiKey Yes Unknown
GraphQL Jobs Jobs with GraphQL No Yes Yes
Jobs2Careers Job aggregator apiKey Yes Unknown
Jooble Job search engine apiKey Yes Unknown
Juju Job search engine apiKey No Unknown
Open Skills Job titles, skills and related jobs data No No Unknown
Reed Job board aggregator apiKey Yes Unknown
The Muse Job board and company profiles apiKey Yes Unknown
Upwork Freelance job board and management system OAuth Yes Unknown
USAJOBS US government job board apiKey Yes Unknown
WhatJobs Job search engine apiKey Yes Unknown
ZipRecruiter Job search app and website apiKey Yes Unknown

⬆ Back to Index

Machine Learning

API Description Auth HTTPS CORS
AI For Thai Free Various Thai AI API apiKey Yes Yes
Clarifai Computer Vision OAuth Yes Unknown
Cloudmersive Image captioning, face recognition, NSFW classification apiKey Yes Yes
Deepcode AI for code review No Yes Unknown
Dialogflow Natural Language Processing apiKey Yes Unknown
EXUDE-API Used for the primary ways for filtering the stopping, stemming words from the text data No Yes Yes
Hirak FaceAPI Face detection, face recognition with age estimation/gender estimation, accurate, no quota limits apiKey Yes Unknown
Imagga Image Recognition Solutions like Tagging, Visual Search, NSFW moderation apiKey Yes Unknown
Inferdo Computer Vision services like Facial detection, Image labeling, NSFW classification apiKey Yes Unknown
IPS Online Face and License Plate Anonymization apiKey Yes Unknown
Irisnet Realtime content moderation API that blocks or blurs unwanted images in real-time apiKey Yes Yes
Keen IO Data Analytics apiKey Yes Unknown
Machinetutors AI Solutions: Video/Image Classification & Tagging, NSFW, Icon/Image/Audio Search, NLP apiKey Yes Yes
MessengerX.io A FREE API for developers to build and monetize personalized ML based chat apps apiKey Yes Yes
NLP Cloud NLP API using spaCy and transformers for NER, sentiments, classification, summarization, and more apiKey Yes Unknown
OpenVisionAPI Open source computer vision API based on open source models No Yes Yes
Perspective NLP API to return probability that if text is toxic, obscene, insulting or threatening apiKey Yes Unknown
Roboflow Universe Pre-trained computer vision models apiKey Yes Yes
SkyBiometry Face Detection, Face Recognition and Face Grouping apiKey Yes Unknown
Time Door A time series analysis API apiKey Yes Yes
Unplugg Forecasting API for timeseries data apiKey Yes Unknown
WolframAlpha Provides specific answers to questions using data and algorithms apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
7digital Api of Music store 7digital OAuth Yes Unknown
AI Mastering Automated Music Mastering apiKey Yes Yes
Audiomack Api of the streaming music hub Audiomack OAuth Yes Unknown
Bandcamp API of Music store Bandcamp OAuth Yes Unknown
Bandsintown Music Events No Yes Unknown
Deezer Music OAuth Yes Unknown
Discogs Music OAuth Yes Unknown
Freesound Music Samples apiKey Yes Unknown
Gaana API to retrieve song information from Gaana No Yes Unknown
Genius Crowdsourced lyrics and music knowledge OAuth Yes Unknown
Genrenator Music genre generator No Yes Unknown
iTunes Search Software products No Yes Unknown
Jamendo Music OAuth Yes Unknown
JioSaavn API to retrieve song information, album meta data and many more from JioSaavn No Yes Unknown
KKBOX Get music libraries, playlists, charts, and perform out of KKBOX's platform OAuth Yes Unknown
KSoft.Si Lyrics API to get lyrics for songs apiKey Yes Unknown
LastFm Music apiKey Yes Unknown
Lyrics.ovh Simple API to retrieve the lyrics of a song No Yes Unknown
Mixcloud Music OAuth Yes Yes
MusicBrainz Music No Yes Unknown
Musixmatch Music apiKey Yes Unknown
Napster Music apiKey Yes Yes
Openwhyd Download curated playlists of streaming tracks (YouTube, SoundCloud, etc...) No Yes No
Phishin A web-based archive of legal live audio recordings of the improvisational rock band Phish apiKey Yes No
Radio Browser List of internet radio stations No Yes Yes
Songkick Music Events apiKey Yes Unknown
Songlink / Odesli Get all the services on which a song is available apiKey Yes Yes
Songsterr Provides guitar, bass and drums tabs and chords No Yes Unknown
SoundCloud With SoundCloud API you can build applications that will give more power to control your content OAuth Yes Unknown
Spotify View Spotify music catalog, manage users' libraries, get recommendations and more OAuth Yes Unknown
TasteDive Similar artist API (also works for movies and TV shows) apiKey Yes Unknown
TheAudioDB Music apiKey Yes Unknown
Vagalume Crowdsourced lyrics and music knowledge apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
apilayer mediastack Free, Simple REST API for Live News & Blog Articles apiKey Yes Unknown
Associated Press Search for news and metadata from Associated Press apiKey Yes Unknown
Chronicling America Provides access to millions of pages of historic US newspapers from the Library of Congress No No Unknown
Currents Latest news published in various news sources, blogs and forums apiKey Yes Yes
Feedbin RSS reader OAuth Yes Unknown
GNews Search for news from various sources apiKey Yes Yes
Graphs for Coronavirus Each Country separately and Worldwide Graphs for Coronavirus. Daily updates No Yes Yes
Inshorts News Provides news from inshorts No Yes Unknown
MarketAux Live stock market news with tagged tickers + sentiment and stats JSON API apiKey Yes Yes
New York Times The New York Times Developer Network apiKey Yes Unknown
News Headlines currently published on a range of news sources and blogs apiKey Yes Unknown
NewsData News data API for live-breaking news and headlines from reputed news sources apiKey Yes Unknown
NewsX Get or Search Latest Breaking News with ML Powered Summaries 🤖 apiKey Yes Unknown
NPR One Personalized news listening experience from NPR OAuth Yes Unknown
Spaceflight News Spaceflight related news 🚀 No Yes Yes
The Guardian Access all the content the Guardian creates, categorised by tags and section apiKey Yes Unknown
The Old Reader RSS reader apiKey Yes Unknown
TheNews Aggregated headlines, top story and live news JSON API apiKey Yes Yes
Trove Search through the National Library of Australia collection of 1000s of digitised newspapers apiKey Yes Unknown

⬆ Back to Index

Open Data

API Description Auth HTTPS CORS
18F Unofficial US Federal Government API Development No No Unknown
API Setu An Indian Government platform that provides a lot of APIS for KYC, business, education & employment No Yes Yes
Archive.org The Internet Archive No Yes No
Black History Facts Contribute or search one of the largest black history fact databases on the web apiKey Yes Yes
BotsArchive JSON formatted details about Telegram Bots available in database No Yes Unknown
Callook.info United States ham radio callsigns No Yes Unknown
CARTO Location Information Prediction apiKey Yes Unknown
CollegeScoreCard.ed.gov Data on higher education institutions in the United States No Yes Unknown
Enigma Public Broadest collection of public data apiKey Yes Yes
French Address Search Address search via the French Government No Yes Unknown
GENESIS Federal Statistical Office Germany OAuth Yes Unknown
Joshua Project People groups of the world with the fewest followers of Christ apiKey Yes Unknown
Kaggle Create and interact with Datasets, Notebooks, and connect with Kaggle apiKey Yes Unknown
LinkPreview Get JSON formatted summary with title, description and preview image for any requested URL apiKey Yes Yes
Lowy Asia Power Index Get measure resources and influence to rank the relative power of states in Asia No Yes Unknown
Microlink.io Extract structured data from any website No Yes Yes
Nasdaq Data Link Stock market data apiKey Yes Unknown
Nobel Prize Open data about nobel prizes and events No Yes Yes
Open Data Minneapolis Spatial (GIS) and non-spatial city data for Minneapolis No Yes No
openAFRICA Large datasets repository of African open data No Yes Unknown
OpenCorporates Data on corporate entities and directors in many countries apiKey Yes Unknown
OpenSanctions Data on international sanctions, crime and politically exposed persons No Yes Yes
PeakMetrics News articles and public datasets apiKey Yes Unknown
Recreation Information Database Recreational areas, federal lands, historic sites, museums, and other attractions/resources(US) apiKey Yes Unknown
Scoop.it Content Curation Service apiKey No Unknown
Socrata Access to Open Data from Governments, Non-profits and NGOs around the world OAuth Yes Yes
Teleport Quality of Life Data No Yes Unknown
Umeå Open Data Open data of the city Umeå in northen Sweden No Yes Yes
Universities List University names, countries and domains No Yes Unknown
University of Oslo Courses, lecture videos, detailed information for courses etc. for the University of Oslo (Norway) No Yes Unknown
UPC database More than 1.5 million barcode numbers from all around the world apiKey Yes Unknown
Urban Observatory The largest set of publicly available real time urban data in the UK No No No
Wikidata Collaboratively edited knowledge base operated by the Wikimedia Foundation OAuth Yes Unknown
Wikipedia Mediawiki Encyclopedia No Yes Unknown
Yelp Find Local Business OAuth Yes Unknown

⬆ Back to Index

Open Source Projects

API Description Auth HTTPS CORS
Countly Countly web analytics No No Unknown
Creative Commons Catalog Search among openly licensed and public domain works OAuth Yes Yes
Datamuse Word-finding query engine No Yes Unknown
Drupal.org Drupal.org No Yes Unknown
Evil Insult Generator Evil Insults No Yes Yes
GitHub Contribution Chart Generator Create an image of your GitHub contributions No Yes Yes
GitHub ReadMe Stats Add dynamically generated statistics to your GitHub profile ReadMe No Yes Yes
Metabase An open source Business Intelligence server to share data and analytics inside your company No Yes Yes
Shields Concise, consistent, and legible badges in SVG and raster format No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
EPO European patent search system api OAuth Yes Unknown
PatentsView API is intended to explore and visualize trends/patterns across the US innovation landscape No Yes Unknown
TIPO Taiwan patent search system api apiKey Yes Unknown
USPTO USA patent api services No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Advice Slip Generate random advice slips No Yes Unknown
Biriyani As A Service Biriyani images placeholder No Yes No
Dev.to Access Forem articles, users and other resources via API apiKey Yes Unknown
Dictum API to get access to the collection of the most inspiring expressions of mankind No Yes Unknown
FavQs.com FavQs allows you to collect, discover and share your favorite quotes apiKey Yes Unknown
FOAAS Fuck Off As A Service No No Unknown
Forismatic Inspirational Quotes No No Unknown
icanhazdadjoke The largest selection of dad jokes on the internet No Yes Unknown
Inspiration Motivational and Inspirational quotes No Yes Yes
kanye.rest REST API for random Kanye West quotes No Yes Yes
kimiquotes Team radio and interview quotes by Finnish F1 legend Kimi Räikkönen No Yes Yes
Medium Community of readers and writers offering unique perspectives on ideas OAuth Yes Unknown
Programming Quotes Programming Quotes API for open source projects No Yes Unknown
Quotable Quotes Quotable is a free, open source quotations API No Yes Unknown
Quote Garden REST API for more than 5000 famous quotes No Yes Unknown
quoteclear Ever-growing list of James Clear quotes from the 3-2-1 Newsletter No Yes Yes
Quotes on Design Inspirational Quotes No Yes Unknown
Stoicism Quote Quotes about Stoicism No Yes Unknown
They Said So Quotes Quotes Trusted by many fortune brands around the world No Yes Unknown
Traitify Assess, collect and analyze Personality No Yes Unknown
Udemy(instructor) API for instructors on Udemy apiKey Yes Unknown
Vadivelu HTTP Codes On demand HTTP Codes with images No Yes No
Zen Quotes Large collection of Zen quotes for inspiration No Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
Abstract Phone Validation Validate phone numbers globally apiKey Yes Yes
apilayer numverify Phone number validation apiKey Yes Unknown
Cloudmersive Validate Validate international phone numbers apiKey Yes Yes
Phone Specification Rest Api for Phone specifications No Yes Yes
Veriphone Phone number validation & carrier lookup apiKey Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
apilayer screenshotlayer URL 2 Image No Yes Unknown
APITemplate.io Dynamically generate images and PDFs from templates with a simple API apiKey Yes Yes
Bruzu Image generation with query string apiKey Yes Yes
CheetahO Photo optimization and resize apiKey Yes Unknown
Dagpi Image manipulation and processing apiKey Yes Unknown
Duply Generate, Edit, Scale and Manage Images and Videos Smarter & Faster apiKey Yes Yes
DynaPictures Generate Hundreds of Personalized Images in Minutes apiKey Yes Yes
Flickr Flickr Services OAuth Yes Unknown
Getty Images Build applications using the world's most powerful imagery OAuth Yes Unknown
Gfycat Jiffier GIFs OAuth Yes Unknown
Giphy Get all your gifs apiKey Yes Unknown
Google Photos Integrate Google Photos with your apps or devices OAuth Yes Unknown
Image Upload Image Optimization apiKey Yes Unknown
Imgur Images OAuth Yes Unknown
Imsea Free image search No Yes Unknown
Lorem Picsum Images from Unsplash No Yes Unknown
ObjectCut Image Background removal apiKey Yes Yes
Pexels Free Stock Photos and Videos apiKey Yes Yes
PhotoRoom Remove background from images apiKey Yes Unknown
Pixabay Photography apiKey Yes Unknown
PlaceKeanu Resizable Keanu Reeves placeholder images with grayscale and young Keanu options No Yes Unknown
Readme typing SVG Customizable typing and deleting text SVG No Yes Unknown
Remove.bg Image Background removal apiKey Yes Unknown
ReSmush.it Photo optimization No No Unknown
shutterstock Stock Photos and Videos OAuth Yes Unknown
Sirv Image management solutions like optimization, manipulation, hosting apiKey Yes Unknown
Unsplash Photography OAuth Yes Unknown
Wallhaven Wallpapers apiKey Yes Unknown
Webdam Images OAuth Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Codeforces Get access to Codeforces data apiKey Yes Unknown
Hackerearth For compiling and running code in several languages apiKey Yes Unknown
Judge0 CE Online code execution system apiKey Yes Unknown
KONTESTS For upcoming and ongoing competitive coding contests No Yes Unknown
Mintlify For programmatically generating documentation for code apiKey Yes Yes

⬆ Back to Index

Science & Math

API Description Auth HTTPS CORS
arcsecond.io Multiple astronomy data sources No Yes Unknown
arXiv Curated research-sharing platform: physics, mathematics, quantitative finance, and economics No Yes Unknown
CORE Access the world's Open Access research papers apiKey Yes Unknown
GBIF Global Biodiversity Information Facility No Yes Yes
iDigBio Access millions of museum specimens from organizations around the world No Yes Unknown
inspirehep.net High Energy Physics info. system No Yes Unknown
isEven (humor) Check if a number is even No Yes Unknown
ISRO ISRO Space Crafts Information No Yes No
ITIS Integrated Taxonomic Information System No Yes Unknown
Launch Library 2 Spaceflight launches and events database No Yes Yes
Materials Platform for Data Science Curated experimental data for materials science apiKey Yes No
Minor Planet Center Asterank.com Information No No Unknown
NASA NASA data, including imagery No Yes No
NASA ADS NASA Astrophysics Data System OAuth Yes Yes
Newton Symbolic and Arithmetic Math Calculator No Yes No
Noctua REST API used to access NoctuaSky features No Yes Unknown
Numbers Number of the day, random number, number facts and anything else you want to do with numbers apiKey Yes No
Numbers Facts about numbers No No No
Ocean Facts Facts pertaining to the physical science of Oceanography No Yes Unknown
Open Notify ISS astronauts, current location, etc No No No
Open Science Framework Repository and archive for study designs, research materials, data, manuscripts, etc No Yes Unknown
Purple Air Real Time Air Quality Monitoring No Yes Unknown
Remote Calc Decodes base64 encoding and parses it to return a solution to the calculation in JSON No Yes Yes
SHARE A free, open, dataset about research and scholarly activities No Yes No
SpaceX Company, vehicle, launchpad and launch data No Yes No
SpaceX GraphQL, Company, Ships, launchpad and launch data No Yes Unknown
Sunrise and Sunset Sunset and sunrise times for a given latitude and longitude No Yes No
Times Adder With this API you can add each of the times introduced in the array sended No Yes No
TLE Satellite information No Yes No
USGS Earthquake Hazards Program Earthquakes data real-time No Yes No
USGS Water Services Water quality and level info for rivers and lakes No Yes No
World Bank World Data No Yes No
xMath Random mathematical expressions No Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
Application Environment Verification Android library and API to verify the safety of user devices, detect rooted devices and other risks apiKey Yes Yes
BinaryEdge Provide access to BinaryEdge 40fy scanning platform apiKey Yes Yes
BitWarden Best open-source password manager OAuth Yes Unknown
Botd Botd is a browser library for JavaScript bot detection apiKey Yes Yes
Bugcrowd Bugcrowd API for interacting and tracking the reported issues programmatically apiKey Yes Unknown
Censys Search engine for Internet connected host and devices apiKey Yes No
Classify Encrypting & decrypting text messages No Yes Yes
Complete Criminal Checks Provides data of offenders from all U.S. States and Pureto Rico apiKey Yes Yes
CRXcavator Chrome extension risk scoring apiKey Yes Unknown
Dehash.lt Hash decryption MD5, SHA1, SHA3, SHA256, SHA384, SHA512 No Yes Unknown
EmailRep Email address threat and risk prediction No Yes Unknown
Escape An API for escaping different kind of queries No Yes No
FilterLists Lists of filters for adblockers and firewalls No Yes Unknown
FingerprintJS Pro Fraud detection API offering highly accurate browser fingerprinting apiKey Yes Yes
FraudLabs Pro Screen order information using AI to detect frauds apiKey Yes Unknown
FullHunt Searchable attack surface database of the entire internet apiKey Yes Unknown
GitGuardian Scan files for secrets (API Keys, database credentials) apiKey Yes No
GreyNoise Query IPs in the GreyNoise dataset and retrieve a subset of the full IP context data apiKey Yes Unknown
HackerOne The industry’s first hacker API that helps increase productivity towards creative bug bounty hunting apiKey Yes Unknown
Hashable A REST API to access high level cryptographic functions and methods No Yes Yes
HaveIBeenPwned Passwords which have previously been exposed in data breaches apiKey Yes Unknown
Intelligence X Perform OSINT via Intelligence X apiKey Yes Unknown
LoginRadius Managed User Authentication Service apiKey Yes Yes
Microsoft Security Response Center (MSRC) Programmatic interfaces to engage with the Microsoft Security Response Center (MSRC) No Yes Unknown
Mozilla http scanner Mozilla observatory http scanner No Yes Unknown
Mozilla tls scanner Mozilla observatory tls scanner No Yes Unknown
National Vulnerability Database U.S. National Vulnerability Database No Yes Unknown
Passwordinator Generate random passwords of varying complexities No Yes Yes
PhishStats Phishing database No Yes Unknown
Privacy.com Generate merchant-specific and one-time use credit card numbers that link back to your bank apiKey Yes Unknown
Pulsedive Scan, search and collect threat intelligence data in real-time apiKey Yes Unknown
SecurityTrails Domain and IP related information such as current and historical WHOIS and DNS records apiKey Yes Unknown
Shodan Search engine for Internet connected devices apiKey Yes Unknown
Spyse Access data on all Internet assets and build powerful attack surface management applications apiKey Yes Unknown
Threat Jammer Risk scoring service from curated threat intelligence data apiKey Yes Unknown
UK Police UK Police data No Yes Unknown
Virushee Virushee file/data scanning No Yes Yes
VulDB VulDB API allows to initiate queries for one or more items along with transactional bots apiKey Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Best Buy Products, Buying Options, Categories, Recommendations, Stores and Commerce apiKey Yes Unknown
Digi-Key Retrieve price and inventory of electronic components as well as place orders OAuth Yes Unknown
Dummy Products An api to fetch dummy e-commerce products JSON data with placeholder images apiKey Yes Yes
eBay Sell and Buy on eBay OAuth Yes Unknown
Etsy Manage shop and interact with listings OAuth Yes Unknown
Flipkart Marketplace Product listing management, Order Fulfilment in the Flipkart Marketplace OAuth Yes Yes
Lazada Retrieve product ratings and seller performance metrics apiKey Yes Unknown
Mercadolibre Manage sales, ads, products, services and Shops apiKey Yes Unknown
Octopart Electronic part data for manufacturing, design, and sourcing apiKey Yes Unknown
OLX Poland Integrate with local sites by posting, managing adverts and communicating with OLX users apiKey Yes Unknown
Rappi Manage orders from Rappi's app OAuth Yes Unknown
Shopee Shopee's official API for integration of various services from Shopee apiKey Yes Unknown
Tokopedia Tokopedia's Official API for integration of various services from Tokopedia OAuth Yes Unknown
WooCommerce WooCommerce REST APIS to create, read, update, and delete data on wordpress website in JSON format apiKey Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
4chan Simple image-based bulletin board dedicated to a variety of topics No Yes Yes
Ayrshare Social media APIs to post, get analytics, and manage multiple users social media accounts apiKey Yes Yes
aztro Daily horoscope info for yesterday, today, and tomorrow No Yes Unknown
Blogger The Blogger APIs allows client applications to view and update Blogger content OAuth Yes Unknown
Cisco Spark Team Collaboration Software OAuth Yes Unknown
Dangerous Discord Database Database of malicious Discord accounts apiKey Yes Unknown
Discord Make bots for Discord, integrate Discord onto an external platform OAuth Yes Unknown
Disqus Communicate with Disqus data OAuth Yes Unknown
Doge-Meme Top meme posts from r/dogecoin which include 'Meme' flair No Yes Yes
Facebook Facebook Login, Share on FB, Social Plugins, Analytics and more OAuth Yes Unknown
Foursquare Interact with Foursquare users and places (geolocation-based checkins, photos, tips, events, etc) OAuth Yes Unknown
Fuck Off as a Service Asks someone to fuck off No Yes Unknown
Full Contact Get Social Media profiles and contact Information OAuth Yes Unknown
HackerNews Social news for CS and entrepreneurship No Yes Unknown
Hashnode A blogging platform built for developers No Yes Unknown
Instagram Instagram Login, Share on Instagram, Social Plugins and more OAuth Yes Unknown
Kakao Kakao Login, Share on KakaoTalk, Social Plugins and more OAuth Yes Unknown
Lanyard Retrieve your presence on Discord through an HTTP REST API or WebSocket No Yes Yes
Line Line Login, Share on Line, Social Plugins and more OAuth Yes Unknown
LinkedIn The foundation of all digital integrations with LinkedIn OAuth Yes Unknown
Meetup.com Data about Meetups from Meetup.com apiKey Yes Unknown
Microsoft Graph Access the data and intelligence in Microsoft 365, Windows 10, and Enterprise Mobility OAuth Yes Unknown
NAVER NAVER Login, Share on NAVER, Social Plugins and more OAuth Yes Unknown
Open Collective Get Open Collective data No Yes Unknown
Pinterest The world's catalog of ideas OAuth Yes Unknown
Product Hunt The best new products in tech OAuth Yes Unknown
Reddit Homepage of the internet OAuth Yes Unknown
Revolt Revolt open source Discord alternative apiKey Yes Unknown
Saidit Open Source Reddit Clone OAuth Yes Unknown
Slack Team Instant Messaging OAuth Yes Unknown
TamTam Bot API to interact with TamTam apiKey Yes Unknown
Telegram Bot Simplified HTTP version of the MTProto API for bots apiKey Yes Unknown
Telegram MTProto Read and write Telegram data OAuth Yes Unknown
Telegraph Create attractive blogs easily, to share apiKey Yes Unknown
TikTok Fetches user info and user's video posts on TikTok platform OAuth Yes Unknown
Trash Nothing A freecycling community with thousands of free items posted every day OAuth Yes Yes
Tumblr Read and write Tumblr Data OAuth Yes Unknown
Twitch Game Streaming API OAuth Yes Unknown
Twitter Read and write Twitter data OAuth Yes No
vk Read and write vk data OAuth Yes Unknown

⬆ Back to Index

Sports & Fitness

API Description Auth HTTPS CORS
API-FOOTBALL Get information about Football Leagues & Cups apiKey Yes Yes
ApiMedic ApiMedic offers a medical symptom checker API primarily for patients apiKey Yes Unknown
balldontlie Balldontlie provides access to stats data from the NBA No Yes Yes
Canadian Football League (CFL) Official JSON API providing real-time league, team and player statistics about the CFL apiKey Yes No
City Bikes City Bikes around the world No Yes Unknown
Cloudbet Official Cloudbet API provides real-time sports odds and betting API to place bets programmatically apiKey Yes Yes
CollegeFootballData.com Unofficial detailed American college football statistics, records, and results API apiKey Yes Unknown
Ergast F1 F1 data from the beginning of the world championships in 1950 No Yes Unknown
Fitbit Fitbit Information OAuth Yes Unknown
Football A simple Open Source Football API to get squads’ stats, best scorers and more X-Mashape-Key Yes Unknown
Football (Soccer) Videos Embed codes for goals and highlights from Premier League, Bundesliga, Serie A and many more No Yes Yes
Football Standings Display football standings e.g epl, la liga, serie a etc. The data is based on espn site No Yes Yes
Football-Data Football data with matches info, players, teams, and competitions X-Mashape-Key Yes Unknown
JCDecaux Bike JCDecaux's self-service bicycles apiKey Yes Unknown
MLB Records and Stats Current and historical MLB statistics No No Unknown
NBA Data All NBA Stats DATA, Games, Livescore, Standings, Statistics apiKey Yes Unknown
NBA Stats Current and historical NBA Statistics No Yes Unknown
NHL Records and Stats NHL historical data and statistics No Yes Unknown
Oddsmagnet Odds history from multiple UK bookmakers No Yes Yes
OpenLigaDB Crowd sourced sports league results No Yes Yes
Premier League Standings All Current Premier League Standings and Statistics apiKey Yes Unknown
Sport Data Get sports data from all over the world apiKey Yes Unknown
Sport List & Data List of and resources related to sports No Yes Yes
Sport Places Crowd-source sports places around the world No Yes No
Sport Vision Identify sport, brands and gear in an image. Also does image sports captioning apiKey Yes Yes
Sportmonks Cricket Live cricket score, player statistics and fantasy API apiKey Yes Unknown
Sportmonks Football Football score/schedule, news api, tv channels, stats, history, display standing e.g. epl, la liga apiKey Yes Unknown
Squiggle Fixtures, results and predictions for Australian Football League matches No Yes Yes
Strava Connect with athletes, activities and more OAuth Yes Unknown
SuredBits Query sports data, including teams, players, games, scores and statistics No No No
TheSportsDB Crowd-Sourced Sports Data and Artwork apiKey Yes Yes
Tredict Get and set activities, health data and more OAuth Yes Unknown
Wger Workout manager data as exercises, muscles or equipment apiKey Yes Unknown

⬆ Back to Index

Test Data

API Description Auth HTTPS CORS
Bacon Ipsum A Meatier Lorem Ipsum Generator No Yes Unknown
Dicebear Avatars Generate random pixel-art avatars No Yes No
English Random Words Generate English Random Words with Pronunciation No Yes No
FakeJSON Service to generate test and fake data apiKey Yes Yes
FakerAPI APIs collection to get fake data No Yes Yes
FakeStoreAPI Fake store rest API for your e-commerce or shopping website prototype No Yes Unknown
GeneradorDNI Data generator API. Profiles, vehicles, banks and cards, etc apiKey Yes Unknown
ItsThisForThat Generate Random startup ideas No Yes No
JSONPlaceholder Fake data for testing and prototyping No No Unknown
Loripsum The "lorem ipsum" generator that doesn't suck No No Unknown
Mailsac Disposable Email apiKey Yes Unknown
Metaphorsum Generate demo paragraphs giving number of words and sentences No No Unknown
Mockaroo Generate fake data to JSON, CSV, TXT, SQL and XML apiKey Yes Unknown
QuickMocker API mocking tool to generate contextual, fake or random data No Yes Yes
Random Data Random data generator No Yes Unknown
Randommer Random data generator apiKey Yes Yes
RandomUser Generates and list user data No Yes Unknown
RoboHash Generate random robot/alien avatars No Yes Unknown
Spanish random names Generate spanish names (with gender) randomly No Yes Unknown
Spanish random words Generate spanish words randomly No Yes Unknown
This Person Does not Exist Generates real-life faces of people who do not exist No Yes Unknown
Toolcarton Generate random testimonial data No Yes Unknown
UUID Generator Generate UUIDs No Yes No
What The Commit Random commit message generator No No Yes
Yes No Generate yes or no randomly No Yes Unknown

⬆ Back to Index

Text Analysis

API Description Auth HTTPS CORS
Code Detection API Detect, label, format and enrich the code in your app or in your data pipeline OAuth Yes Unknown
apilayer languagelayer Language Detection JSON API supporting 173 languages OAuth Yes Unknown
Aylien Text Analysis A collection of information retrieval and natural language APIs apiKey Yes Unknown
Cloudmersive Natural Language Processing Natural language processing and text analysis apiKey Yes Yes
Detect Language Detects text language apiKey Yes Unknown
ELI Natural Language Processing Tools for Thai Language apiKey Yes Unknown
Google Cloud Natural Natural language understanding technology, including sentiment, entity and syntax analysis apiKey Yes Unknown
Hirak OCR Image to text -text recognition- from image more than 100 language, accurate, unlimited requests apiKey Yes Unknown
Hirak Translation Translate between 21 of most used languages, accurate, unlimited requests apiKey Yes Unknown
Lecto Translation Translation API with free tier and reasonable prices apiKey Yes Yes
LibreTranslate Translation tool with 17 available languages No Yes Unknown
Semantria Text Analytics with sentiment analysis, categorization & named entity extraction OAuth Yes Unknown
Sentiment Analysis Multilingual sentiment analysis of texts from different sources apiKey Yes Yes
Tisane Text Analytics with focus on detection of abusive content and law enforcement applications OAuth Yes Yes
Watson Natural Language Understanding Natural language processing for advanced text analysis OAuth Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
Aftership API to update, manage and track shipment efficiently apiKey Yes Yes
Correios Integration to provide information and prepare shipments using Correio's services apiKey Yes Unknown
Pixela API for recording and tracking habits or effort, routines X-Mashape-Key Yes Yes
PostalPinCode API for getting Pincode details in India No Yes Unknown
Postmon An API to query Brazilian ZIP codes and orders easily, quickly and free No No Unknown
PostNord Provides information about parcels in transport for Sweden and Denmark apiKey No Unknown
UPS Shipment and Address information apiKey Yes Unknown
WeCanTrack Automatically place subids in affiliate links to attribute affiliate conversions to click data apiKey Yes Yes
WhatPulse Small application that measures your keyboard/mouse usage No Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
ADS-B Exchange Access real-time and historical data of any and all airborne aircraft No Yes Unknown
airportsapi Get name and website-URL for airports by ICAO code No Yes Unknown
AIS Hub Real-time data of any marine and inland vessel equipped with AIS tracking system apiKey No Unknown
Amadeus for Developers Travel Search - Limited usage OAuth Yes Unknown
apilayer aviationstack Real-time Flight Status & Global Aviation Data API OAuth Yes Unknown
AviationAPI FAA Aeronautical Charts and Publications, Airport Information, and Airport Weather No Yes No
AZ511 Access traffic data from the ADOT API apiKey Yes Unknown
Bay Area Rapid Transit Stations and predicted arrivals for BART apiKey No Unknown
BC Ferries Sailing times and capacities for BC Ferries No Yes Yes
BIC-Boxtech Container technical detail for the global container fleet OAuth Yes Unknown
BlaBlaCar Search car sharing trips apiKey Yes Unknown
Boston MBTA Transit Stations and predicted arrivals for MBTA apiKey Yes Unknown
Community Transit Transitland API No Yes Unknown
Compare Flight Prices API for comparing flight prices across platforms apiKey Yes Unknown
CTS CTS Realtime API apiKey Yes Yes
Grab Track deliveries, ride fares, payments and loyalty points OAuth Yes Unknown
GraphHopper A-to-B routing with turn-by-turn instructions apiKey Yes Unknown
Icelandic APIs Open APIs that deliver services in or regarding Iceland No Yes Unknown
Impala Hotel Bookings Hotel content, rates and room bookings apiKey Yes No
Izi Audio guide for travellers apiKey Yes Unknown
Land Transport Authority DataMall, Singapore Singapore transport information apiKey No Unknown
Metro Lisboa Delays in subway lines No No No
Navitia The open API for building cool stuff with transport data apiKey Yes Unknown
Open Charge Map Global public registry of electric vehicle charging locations apiKey Yes Yes
OpenSky Network Free real-time ADS-B aviation data No Yes Unknown
Railway Transport for France SNCF public API apiKey Yes Unknown
REFUGE Restrooms Provides safe restroom access for transgender, intersex and gender nonconforming individuals No Yes Unknown
Sabre for Developers Travel Search - Limited usage apiKey Yes Unknown
Schiphol Airport Schiphol apiKey Yes Unknown
Tankerkoenig German realtime gas/diesel prices apiKey Yes Yes
TransitLand Transit Aggregation No Yes Unknown
Transport for Atlanta, US Marta No No Unknown
Transport for Auckland, New Zealand Auckland Transport No Yes Unknown
Transport for Belgium The iRail API is a third-party API for Belgian public transport by train No Yes Yes
Transport for Berlin, Germany Third-party VBB API No Yes Unknown
Transport for Bordeaux, France Bordeaux Métropole public transport and more (France) apiKey Yes Unknown
Transport for Budapest, Hungary Budapest public transport API No Yes Unknown
Transport for Chicago, US Chicago Transit Authority (CTA) apiKey No Unknown
Transport for Czech Republic Czech transport API No Yes Unknown
Transport for Denver, US RTD No No Unknown
Transport for Finland Finnish transport API No Yes Unknown
Transport for Germany Deutsche Bahn (DB) API apiKey No Unknown
Transport for Grenoble, France Grenoble public transport No No No
Transport for Hessen, Germany RMV API (Public Transport in Hessen) No Yes Unknown
Transport for Honolulu, US Honolulu Transportation Information apiKey No Unknown
Transport for Lisbon, Portugal Data about buses routes, parking and traffic apiKey Yes Unknown
Transport for London, England TfL API apiKey Yes Unknown
Transport for Los Angeles, US Data about positions of Metro vehicles in real time and travel their routes No Yes Unknown
Transport for Manchester, England TfGM transport network data apiKey Yes No
Transport for Norway Transport APIs and dataset for Norway No Yes Unknown
Transport for Ottawa, Canada OC Transpo API apiKey Yes Unknown
Transport for Paris, France RATP Open Data API No No Unknown
Transport for Philadelphia, US SEPTA APIs No No Unknown
Transport for Sao Paulo, Brazil SPTrans OAuth No Unknown
Transport for Spain Public trains of Spain No Yes Unknown
Transport for Sweden Public Transport consumer OAuth Yes Unknown
Transport for Switzerland Official Swiss Public Transport Open Data apiKey Yes Unknown
Transport for Switzerland Swiss public transport API No Yes Unknown
Transport for The Netherlands NS, only trains apiKey No Unknown
Transport for The Netherlands OVAPI, country-wide public transport No Yes Unknown
Transport for Toronto, Canada TTC No Yes Unknown
Transport for UK Transport API and dataset for UK apiKey Yes Unknown
Transport for United States NextBus API No No Unknown
Transport for Vancouver, Canada TransLink OAuth Yes Unknown
Transport for Washington, US Washington Metro transport API OAuth Yes Unknown
transport.rest Community maintained, developer-friendly public transport API No Yes Yes
Tripadvisor Rating content for a hotel, restaurant, attraction or destination apiKey Yes Unknown
Uber Uber ride requests and price estimation OAuth Yes Yes
Velib metropolis, Paris, France Velib Open Data API No Yes No

⬆ Back to Index

URL Shorteners

API Description Auth HTTPS CORS
1pt A simple URL shortener No Yes Yes
Bitly URL shortener and link management OAuth Yes Unknown
CleanURI URL shortener service No Yes Yes
ClickMeter Monitor, compare and optimize your marketing links apiKey Yes Unknown
Clico URL shortener service apiKey Yes Unknown
Cutt.ly URL shortener service apiKey Yes Unknown
Drivet URL Shortener Shorten a long URL easily and fast No Yes Unknown
Free Url Shortener Free URL Shortener offers a powerful API to interact with other sites No Yes Unknown
Git.io Git.io URL shortener No Yes Unknown
GoTiny A lightweight URL shortener, focused on ease-of-use for the developer and end-user No Yes Yes
Kutt Free Modern URL Shortener apiKey Yes Yes
Mgnet.me Torrent URL shorten API No Yes No
owo A simple link obfuscator/shortener No Yes Unknown
Rebrandly Custom URL shortener for sharing branded links apiKey Yes Unknown
Short Link Short URLs support so many domains No Yes Unknown
Shrtcode URl Shortener with multiple Domains No Yes Yes
Shrtlnk Simple and efficient short link creation apiKey Yes Yes
TinyURL Shorten long URLs apiKey Yes No
UrlBae Simple and efficient short link creation apiKey Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
Brazilian Vehicles and Prices Vehicles information from Fundação Instituto de Pesquisas Econômicas - Fipe No Yes No
Helipaddy sites Helicopter and passenger drone landing site directory, Helipaddy data and much more apiKey Yes Unknown
Kelley Blue Book Vehicle info, pricing, configuration, plus much more apiKey Yes No
Mercedes-Benz Telematics data, remotely access vehicle functions, car configurator, locate service dealers apiKey Yes No
NHTSA NHTSA Product Information Catalog and Vehicle Listing No Yes Unknown
Smartcar Lock and unlock vehicles and get data like odometer reading and location. Works on most new cars OAuth Yes Yes

⬆ Back to Index


API Description Auth HTTPS CORS
An API of Ice And Fire Game Of Thrones API No Yes Unknown
Bob's Burgers Bob's Burgers API No Yes Yes
Breaking Bad Breaking Bad API No Yes Unknown
Breaking Bad Quotes Some Breaking Bad quotes No Yes Unknown
Catalogopolis Doctor Who API No Yes Unknown
Catch The Show REST API for next-episode.net No Yes Unknown
Czech Television TV programme of Czech TV No No Unknown
Dailymotion Dailymotion Developer API OAuth Yes Unknown
Dune A simple API which provides you with book, character, movie and quotes JSON data No Yes Yes
Final Space Final Space API No Yes Yes
Game of Thrones Quotes Some Game of Thrones quotes No Yes Unknown
Harry Potter Charactes Harry Potter Characters Data with with imagery No Yes Unknown
IMDb-API API for receiving movie, serial and cast information apiKey Yes Unknown
IMDbOT Unofficial IMDb Movie / Series Information No Yes Yes
JSON2Video Create and edit videos programmatically: watermarks,resizing,slideshows,voice-over,text animations apiKey Yes No
Lucifer Quotes Returns Lucifer quotes No Yes Unknown
MCU Countdown A Countdown to the next MCU Film No Yes Yes
Motivational Quotes Random Motivational Quotes No Yes Unknown
Movie Quote Random Movie and Series Quotes No Yes Yes
Open Movie Database Movie information apiKey Yes Unknown
Owen Wilson Wow API for actor Owen Wilson's "wow" exclamations in movies No Yes Yes
Ron Swanson Quotes Television No Yes Unknown
Simkl Movie, TV and Anime data apiKey Yes Unknown
STAPI Information on all things Star Trek No No No
Stranger Things Quotes Returns Stranger Things quotes No Yes Unknown
Stream Czech internet television, films, series and online videos for free No Yes No
Stromberg Quotes Returns Stromberg quotes and more No Yes Unknown
SWAPI All the Star Wars data you've ever wanted No Yes Yes
SWAPI All things Star Wars No Yes Yes
SWAPI GraphQL Star Wars GraphQL API No Yes Unknown
The Lord of the Rings The Lord of the Rings API apiKey Yes Unknown
The Vampire Diaries TV Show Data apiKey Yes Yes
ThronesApi Game Of Thrones Characters Data with imagery No Yes Unknown
TMDb Community-based movie data apiKey Yes Unknown
TrailerAddict Easily embed trailers from TrailerAddict apiKey No Unknown
Trakt Movie and TV Data apiKey Yes Yes
TVDB Television data apiKey Yes Unknown
TVMaze TV Show Data No No Unknown
uNoGS Unofficial Netflix Online Global Search, Search all netflix regions in one place apiKey Yes Yes
Vimeo Vimeo Developer API OAuth Yes Unknown
Watchmode API for finding out the streaming availability of movies & shows apiKey Yes Unknown
Web Series Quotes Generator API generates various Web Series Quote Images No Yes Yes
YouTube Add YouTube functionality to your sites and apps OAuth Yes Unknown

⬆ Back to Index


API Description Auth HTTPS CORS
7Timer! Weather, especially for Astroweather No No Unknown
AccuWeather Weather and forecast data apiKey No Unknown
Aemet Weather and forecast data from Spain apiKey Yes Unknown
apilayer weatherstack Real-Time & Historical World Weather Data API apiKey Yes Unknown
APIXU Weather apiKey Yes Unknown
AQICN Air Quality Index Data for over 1000 cities apiKey Yes Unknown
AviationWeather NOAA aviation weather forecasts and observations No Yes Unknown
ColorfulClouds Weather apiKey Yes Yes
Euskalmet Meteorological data of the Basque Country apiKey Yes Unknown
Foreca Weather OAuth Yes Unknown
HG Weather Provides weather forecast data for cities in Brazil apiKey Yes Yes
Hong Kong Obervatory Provide weather information, earthquake information, and climate data No Yes Unknown
MetaWeather Weather No Yes No
Meteorologisk Institutt Weather and climate data User-Agent Yes Unknown
Micro Weather Real time weather forecasts and historic data apiKey Yes Unknown
ODWeather Weather and weather webcams No No Unknown
Oikolab 70+ years of global, hourly historical and forecast weather data from NOAA and ECMWF apiKey Yes Yes
Open-Meteo Global weather forecast API for non-commercial use No Yes Yes
openSenseMap Data from Personal Weather Stations called senseBoxes No Yes Yes
OpenUV Real-time UV Index Forecast apiKey Yes Unknown
OpenWeatherMap Weather apiKey Yes Unknown
QWeather Location-based weather data apiKey Yes Yes
RainViewer Radar data collected from different websites across the Internet No Yes Unknown
Storm Glass Global marine weather from multiple sources apiKey Yes Yes
Tomorrow Weather API Powered by Proprietary Technology apiKey Yes Unknown
US Weather US National Weather Service No Yes Yes
Visual Crossing Global historical and weather forecast data apiKey Yes Yes
weather-api A RESTful free API to check the weather No Yes No
WeatherAPI Weather API with other stuff like Astronomy and Geolocation API apiKey Yes Yes
Weatherbit Weather apiKey Yes Unknown
Yandex.Weather Assesses weather condition in specific locations apiKey Yes No

⬆ Back to Index


MIT (c) 2022 public-apis